There are 2 basic ways to achieve Anti-Aliasing:
Increase the sample rate (used e.g. in MSAA, SSAA and custom modes like EQAA and CSAA)
Blur the edges/contrasts (used e.g. in MLAA, FXAA and SMAA), also called Post-AA or Post-Processing.
The simplest way to increase the sample rate is called FSAA(Full Screen AA), SSAA(Super Sampling AA)1 or Downsampling2. In this case, an increased amount of samples are used and the color of each Pixel is calculated using the values of the samples inside it. This results in Pixels that have a mixture of the colors that are actually inside it.
This is arguably the best form of AA: textures get sharper because of the higher sample rate, the Aliasing is greatly reduced and the image is very still. Usually, there should be no blur either. The disadvantage of this mode is the performance needed: its the greatest of all AA modes and only enthusiast rigs, often with mutliple GPUs have the power to use this mode in modern games.
1the right name for this method is OGSSAA aka ordered grid super sampling AA. other method like SGSSAA or RGSSAA dont samples ordered alongside the axes
2Downsampling works slightly different and is more of trick when SSAA doesnt work: the whole frame is rendered in a higher resolution and then downfiltered.
MSAA (Multi Sampling AA) reduces the performance needed compared to SSAA. MSAA detects the edges of polygons and only increases the number of samples there.
The main advantage is that it offeres AA that does not blur and uses less performance than SSAA. the disadvantages are that some deferred-rendering engines (like UE3 and most other PS360-era engines) have problems using MSAA and often have subpar results. It also doesnt stop the aliasing of alpha-textures. Some methods like alpha-to-coverage can help smooth alpha textures using MSAA.
edit: The technical explenation of MSAA was a simplification. A more in-depth explanation can be read here. thanks to /u/fb39ca4 for the english source.
EQAA(Enhanced Quality AA) and CSAA(Coverage Sample AA) try to increase the quality of MSAA. The actual way it does it (increasing the number of coverage-samples while the number of color/depth/stencil-samples remain the same) is complicated, a detailed explenaition can be found here.
MLAA(Morphological AA) and FXAA(Fast Aproximate AA) are post AA modes that use blur filters. First, it detects contrasts ("edges") in the frame and then blurres it along the gradient.
This results in higly reduces visible "jaggies" that also coveres alpha-texturs, but it also blurs everything, including textures. It is also the cheapest form of AA and often used in console version of games.
Personally I dont really like this mode of AA. If you want cheap AA, look at SMAA.
SMAA is an AA mode based on the Post-AA blur filter of MLAA (and FXAA). The alisasing "detection" is upgraded and is closer to the detection used in MSAA then the detection used in MLAA and FXAA. The result is that SMAA still remains very cheap, still smoothes alpha-tectures and still greatly reduces the visible "jaggies", but doesnt blur the image as much.
Personally I think this is one of the best AA modes available. Forcing a slight form of SMAA via driver or tools like RadeonPro or nVidia Inspector combined with traditional MSAA/SSAA will resilt in one of the best results possible.
TXAA(Temporal AA) is a very complex form of AA. It is not a post-AA altough it still blurs because of the downsampling method used. The information we have is also vague, so I would like to stop commenting on the technical side here.
The imlementation of TXAA varies from game to game and version to version of TXAA, so a general statement is hardly possible. What can be said is that it a) uses much more performance than FXAA, MLAA and SMAA, b) the reducement of "jaggies" is one of the best of all AA modes and c) everything blurs.
Because it often blures much more than MLAA or FXAA it is ihmo not that great of a mode. If the sampling rate used internally for TXAA is upgraded to SSAA (it is based on MSAA) the result can be quite good, but it needs a **** ton of additional performance most rigs dont have. If used on very high resolulutions (4K or higher), it might be acceptable too. Overall a mode that might be more usefull in the future and/or in some special games and/or after some adjustments.
