Data Science at the Command Line

Obtain, scrub, explore, and model data with unix power tools.

Description

The unix command line, although invented decades ago, is an amazing environment for efficiently performing tedious but essential data science tasks. By combining small, powerful, command-line tools (like parallel, jq, and csvkit), you can quickly scrub and explore your data and hack together prototypes.

This hands-on workshop is based on the O’Reilly book Data Science at the Command Line, written by our CEO Jeroen Janssens. You’ll learn how to build fast data pipelines, how to leverage R and Python at the command line, and how to quickly visualise data. No prior knowledge about the unix command line is required.

By the end of this workshop you will have a solid understanding of how to integrate the command line in your data science workflow. Even if you’re already comfortable processing data with, for example, R or Python, being able to also leverage the power of the command line can make you a more effective and efficient data scientist.

Schedule

Day 1

  • Introduction
    • What is the command line?
    • Why learn the command line for doing data science?
    • A real-world data science use case
    • Getting up and running with the Docker image
  • Essential concepts of the unix command line
    • Running command-line tools
    • Combining command-line tools
    • Redirecting input and output
    • Working with files
    • Getting help
  • Obtaining data from logs, spreadsheets, and databases
  • Downloading data from the Internet and accessing APIs using curl
  • Transforming data with filters such as cut, paste, grep, and sed
  • Processing other data formats efficiently
    • JSON with jq
    • CSV with csvkit
    • HTML with pup
    • XML with xmlstarlet

Day 2

  • Running R from the command line
  • Visualising data from the command line
    • Scatter plot
    • Histogram
    • Bar chart
    • Geographic visualisation
  • Parallelising and distributing data-intensive pipelines
  • Creating reusable command-line tools
    • Automate things in a Bash script
    • Convert your existing code to a command-line tool
    • Processing arguments
    • Working with streaming data
  • Applying machine learning
    • Outlier detection
    • Dimensionality reduction
    • Classification
    • Regression
  • Conclusion

What You Will Learn

  • Automate tedious tasks
  • Parallelise and distribute your tasks to multiple cores and machines
  • Convert your existing code to reusable command-line tools
  • Easily inspect, transform, and visualise data
  • Apply a variety of supervised and unsupervised machine learning algorithms

Requirements

Participants are kindly requested to have the following items installed prior to the start of the workshop:

  • Docker Desktop for Windows or for Mac or for Ubuntu
  • The docker image, by running: docker pull datascienceworkshops/data-science-at-the-command-line

Clients

We have previously delivered this workshop (or a custom training based on this workshop) at the following organisations:

Teradata
SURFnet
Snow
Amazon
Accenture
Social Point
Container Solutions
The New York Times
Prezi

Photos and Testimonials

Sanne Bouwman
Data Scientist, Teradata

Great workshop! Very well done and very useful information delivered in an excellent and interactive manner. Jeroen anticipated very well on the different knowledge levels within the group. I would highly recommend the Data Science at the Command Line workshop to anyone that is interested in either kickstarting their command-line experiences or improving their data science with Unix power tools.

Joost van Dijk
Manager Middleware Services, SURFnet

As a seasoned UNIX command line adept, I didn’t expect to learn much from a Data Science at the Command Line workshop. I was wrong! Over the years, many new tools have become available that I didn’t know about, and that can be combined with traditional tools in new ways.

Since attending the workshop, I have been able to simplify and improve the efficiency of many of the scripts I use on a daily basis. Recommended for anyone working from the command line, newbies and ninjas alike!

Marc Canaleta

Besides demonstrating a good knowledge and experience in command-line tools for data science, the instructor had very good training skills, clear communication, and managed to adapt the level of the training to the level of the audience, which is not always easy!

One-day hands-on workshop
Introduction to Linux and the Bash Shell
We’re currently writing the description of this course. Please contact us for more information.
€ 590 per person
One-day hands-on workshop
Mastering Version Control with Git and Github
We’re currently writing the description of this course. Please contact us for more information.
€ 590 per person