Data Development Environment – Mac

A first step to developing with modern technologies such as Big Data systems and NoSQL systems is getting your development environment setup. I like to have many of the tools available locally on my laptop so I can feel free to experiment without breaking a shared server or running up a large bill on the cloud platform used to host the machine.  Checkout my previous post on setting up on Windows to read a little about what I like about Python and Sublime Text.  For this post, let’s just walk through the tools I found myself installing on my Mac to do Python data development (this excludes Scala setup).

  • Python installed by default – using Python 2.7.10
  • install Homebrew – http://brew.sh/
  • install Developer Tools – on command line type git and follow prompts to install developer tools (http://www.cnet.com/how-to/install-command-line-developer-tools-in-os-x/)
  • install pip – sudo easy_install pip
  • install SublimeText2
  • install PyCharm
  • install several things using Homebrew (type at command line):
    • brew update
    • brew install wget
    • brew install gcc
    • brew install apache-spark
  • pip install virtualenv
    • (then use virtualenv and virtualenvwrapper for most things python)
  • create a virtual environment for data-eng and install
    • pip install pandas
    • pip install request
    • …..(a lot more, might add to this list later with some of the heavily used ones)

Hopefully that helps you with getting started, feel free to leave comments if you hit errors along the way and if I’ve dealt with similar errors I will give you some tips.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s