Berkeleydir
Berkeleydir enables you to easily find people in the Berkeley directory. (code)
Why build something new?
Currently the way to find people at UC Berkeley is to use directory.berkeley.edu. This service wasn't that intuitive for me to use.
- Slow searching - Sometimes I didn't know the spelling of a name of someone (e.g. Jon or John). This required multiple searches where you have to keep clicking the back button.
- Dropdown - It requires you to select what to search for in a certain format (e.g.
Vuerings, Christian). - Non-mobile - Since it's not responsive, using it on a mobile device was a pain.
Features
Aiming for a clean and intuitive design, it adds the following features:
- Autocomplete - Searches as soon as you start typing.
- Responsive - Works well on any device and resolution.
- Search always available - Instead of having to click the back button for a new search, you can use the always available search input.
- Search multiple properties - Currently searches for uid, name and email.
Technologies
- Firebase to save the data
- LDAP for querying users
- AngularJS in the front-end
- Node.js for crawling and exposing the API
Challenges
Firebase querying - Setting up Firebase with AngularJS was pretty straightforward, but querying was somewhat of a pain. There's currently no way to do database like queries with Firebase.
Speed - Making it fast was of utter importance and the hardest part. In the beginning I loaded the more than 1,000,000 users in the browser. This made it extremely slow and not very scalable. Now it caches it in Node.js and provides 10 results at a time.
Scraping - In the first iteration I scraped the web version of the Berkeley directory at directory.berkeley.edu. This was incredibly slow and took 40+ hours to complete. Now we use LDAP to get the data and takes approximately 1 hour to complete.
LDAP with Node.js - Thanks to the great documentation on the Berkeley LDAP service, it was very straightforward to connect to it with ldapsearch.
ldapsearch -H ldap://ldap.berkeley.edu -x -b 'ou=people,dc=berkeley,dc=edu' objectclass=*
However, connecting to LDAP with ldapjs, a great LDAP library for Node.js, proved to be a bit harder. Thanks to Mark Cavage for explaining that ldapjs search scopes are backwards. In the end, the only change was to set the scope to sub instead of using the default base.
Future
As with any project, there are always improvements to be made:
- University API - Having a RESTful API for this on developer.berkeley.edu.
- Images - Support for images. Already tried using gravatar for this but almost no one at Berkeley has their email linked with the that service.
- Infinite scroll - Currently it only shows 10 results per search.
If you would like to make any changes yourself, feel free to make a pull request on christianv/berkeleydir.
I love to share interesting ideas.