Headaches are common in early pregnancy, especially in the first trimester when hormones are racing to unprecedented levels; the problem is there are many hormonal surges happening at once. Headaches tend to fizzle out after the first trimester, either because the surges stabilize or because you've begun to deal better with the headaches or the stress that causes them.
Your doctor is limited as to what she can offer to combat a headache or migraine during your pregnancy. Home remedies that may help include rest, increasing your water intake, a cold compress, a soothing bath, or a massage.
You can also speak with your doctor about taking acetaminophen. Although there is a mild chance of liver-toxicity, as long as you follow your doctor's dosage instructions, you should be safe.
Aspirin and the non-steroidal anti-inflammatories (such as Advil, Nuprin, Motrin) are not safe during pregnancy because they can cause complications and effect blood clotting. If you've taken an Advil instead of a Tylenol, don't panic. The studies don't label a pregnancy exposed to an occasional NSAID as doomed; they just indicate that it's an unwise choice compared to acetaminophen.
For headaches resistant to acetaminophen, there are some safe prescription drugs, though Tylenol 2,Tylenol 3, Vicodin, Loricet, and others can sometimes make stress headaches worse.
If you suffer from migraines during pregnancy, Imitrex is a safe bet (but only under your doctor's supervision). It is Category C drug, which means that it hasn't been shown to harm the fetus, but there hasn't been enough volume of data yet to declare the drug safe beyond a shadow of a doubt. Imitrex exposure before you found out you were pregnant should not cause worry.