by Siddhartha Reddy (@sids) on Friday, 15 June 2012

+36
Vote on this proposal
Status: Confirmed
Section
Big Data Infrastructure & Processing

Session type
Lecture

Technical level
Intermediate

Objective

No, this is not another tutorial on using Solr/ElasticSearch/Sphinx/Lucene. Imagine that none of these existed and you need a search engine for your shiny new eCommerce startup. What would you do? Build your own search engine, of course. I’ll go over what it would take to do that.

Description

I plan to cover:

  1. The basic idea of a Search Engine — 1.1 How is it different from a Database? — 1.2 Inverted Index

  2. Techniques for constructing an Inverted Index — 2.1 Distributed Indexing — 2.2 Dynamic Indexing — 2.3 Index Compression

  3. Scoring and Ranking

  4. Scaling Search

Speaker bio

Siddhartha leads the Search & Product Discovery Team at Flipkart — India's largest catalog of online inventory. Prior to this he was at a startup called Ziva Software, working on Mobile Search and automatic Information Extraction from the Web.

Siddhartha has been working on Information Retrieval and in related areas for almost seven years. He has a deep love for all things Search — everything ranging from User Experience to Relevance Ranking to Scaling Search to Search as a Platform.

Siddhartha loves to work with large scale, highly available systems and tons of data. He takes particular interest in application of Machine Learning techniques in all kinds of applications.