This depends greatly on the context, i.e. the problem domain.
What will you be retrieving and how is your data described? Do you already have a sufficiently expressive and decidable ontology? Does the vocabulary of your ontology describe your data well and covers all important details and finesses? The ontology must be decidable so the retrieval can actually finish. The best (and safest) bet is to use ontology languages such as OWL-DL, OWL Lite or RDF/RDFS (in that order). If all this is solved then you have your TBox. After this you must transfer all data that will be searched to ABox – that is, generate individuals from, say, some SQL database of yours. There’s no universal solution to this problem. Many write their own applications which transfer SQL rows to, for example, OWL-DL. This isn’t very difficult. Essentially you have to read SQL and parse it to XML (OWL-DL is based on XML). That can be done in many computer languages. When you have TBox and ABox you have created knowledge base suitable for retrieval with a reasoner.
Finally, you need ontology and a reasoner that support SPARQL (SPARQL Protocol and RDF Query Language). SPARQL is somewhat similar to SQL (at least superficially in syntax) and can be used to query a knowledge base with a reasoner. The output is essentially a retrieved set of data from ABox, described with vocabulary in TBox, that satisfies conditions defined in SPARQL query. Or in other words, that is your retrieved dataset.
To put it all to work I suggest you, for starters, download Stanford’s Protégé. Protégé distribution comes with Pellet reasoner and natively supports SPARQL tab. Open some sample ontology like family.owl and in Protégé 3.x go: Reasoning menu -> Open SPARQL Query panel. A new panel will open on the bottom of the screen where you can input SPARQL queries and see the retrieved individuals (i.e. data). You can also try SPARQL in DBPedia’s Virtuoso Query Editor interface.