libindexer is a fast indexing / data retrieving library written in C language. It was initially made for a search engine but is still a "work in progress".
Read explanations here
Download library version 1.1 here
A patch was made by JC Dufour using hash tables on disk for a more efficient implementation on huge files (we're talking about 100GB+ files here).
Download hash version 1.1 here
Comments and feedback: Fabien Menemenlis