Its some form of silicone/rubber that the IHS is attached with. Intel use epoxy so don't even try this with an intel cpu!
The best way I have found is to use a double edged razorblade.
Tools required :
Double edged Razor blade (wilkinson sword)
Electrical tape
Foam pad
Steady hand!
optional : IPA
Tape the blade off with electrical tape so that only 5mm of the blade is showing.
Wrap plenty of tape around the unexposed blade to prevent some nasty cuts. This also acts as a shim so you cannot get the blade under the ihs for more than 5mm. DO NOT USE ANYTHING THICKER THAN A RAZOR BLADE! You will most definately hack into the PCB or worse. People have had success using stanleyblades but there are more horror stories with them than without.
Put the CPU complete with IHS on the foam the CPU was supplied with and push it down gently so none of the pins are visable, this will help prevent any of the pins being bent when removing the IHS, it isn't foolproff and care still needs to be taken that you don't bend any pins but it does help.
Now you are set to remove the IHS, if you have some IPA handy then spray the exposed bit of the blade as this will help lubrication and therefore less force is required when cutting the silicone.
Start at one corner, do not try and force the blade all the way in, take your time and cut around the IHS keeping the blade flat and taking care not to cut into the PCB. The first cut is the hardest (akin to cutting through some very hard cheese) and you will gradually find it easier once you have been around the IHS a few times (mine took 6). If the IHS refuses to come off DO NOT FORCE IT OR TRY TO PRISE IT OFF just take your time. You really do not need more than 5mm depth - anymore will risk cutting into the smd or even the core, just keep going around until the blade is moving freely. The IHS should simply drop off as the only thing that will then be holding it on is the thermal paste. Prising it could bend the PCB and possibly crack the core = bad.
Once the IHS is off take a look at the core, a white goo should be visible but if it had a bad contact it will be uneven in thickness. Clean it off with IPA. When fitting a HS to a naked core be VERY carefull not to tighten the core down unevenly and DO NOT use a stock heat sink with a clip on attachment as this will probably chip the core. A screw down HS is your best bet. Apply some AS5 or ceramique to the core and spread it out either using a razor or a credit card very thinly. Attach the HS (don't be tempted to see if you have a good contact and remove it once seated) and screw down the HS half a turn each side to get the best possible spread of pressure to the core. I have tightened my Big typhoon down as hard as it would go and not had any problems, just make sure that the load to the core is even and not lopsided.
There are a lot better guides than mine out on the net, some with pictures and it will do no harm to do a lot of research before hand. One word of warning though is that if you cannot afford to replce the cpu if it is destroyed then do not attempt this as the chances of killing a cpu are high, my first attempt ended in a scratched PCB (used a scalpel instead of razor) and therefore was a 2.7 opty 165 that is now a keyring. Oh...and Razors are sharp just incase you didn't know
For the record my opty 170 now loads at 41c where as before it was 53c. This is at 2.85 with 1.42v on air. Worthwhile yes...risky...definately!