Using MongoDB to store intraday stock data

Back when I was looking around for an efficient way to store intraday stock data I decided to investigate Mongodb. Mongo is a document database (Non-Relational) and the user will have to perform any calculations outside of the server. This is fine if you just want to pull some data into your preferred environment (Python, Matlab, etc.)

Store a MongoDB on an external hard drive

One noticeable characteristic of the MongoDB is its Portability, after it's installed it can be moved or copied to another location. It can be moved, for instance, on to an external hard drive and then just cd into your MongoDB bin and run one more path command for a connection.

Right now my database is in on my windows 7 machine in my users folder, so its located on the path with the prefix C:\Users\jesse

The following 2 commands start the windows server for the MongoDB:

cd C:\Users\jesse\mongodb-win32-x86_64-2008plus-2.4.10\mongodb-win32-x86_64-2008plus-2.4.10\bin

mongod --dbpath C:\Users\jesse\mongo\data\db

If the MongoDB server has started correctly you should see something similar to this in the command window.

If you move your database somewhere else later don’t forget to update the path commands to reflect this.

A MongoDB stores groups of collections.

I set my intraday stock database up so that a stock ticker symbol represents the name of one collection. Then I used the db.collection.update command so if the date and time for this stock already existed it would be updated and if it didn’t already exist then a new datetime.datetime object for that collection would be created.

In this collection each unique id will have 5 documents: open, high, low, close, and volume.

A Python function to update a MongoDB.

Here is a Python function that will take in 1 ticker symbol and 6 arrays of data and insert or update a MongoDB collection.

True is for upsert, a command that creates a new document when no document matches the query criteria. So once again this Python function will create new documents for a certain date and time if none exists and if it does already exist this function will update it with the latest information.