[Fast Data Analysis Using C++ and Python]

Python Data science C++ Bash script Date: Sep 2018

C++ and Python code that was applied to analyse data from the Grow-When-Required Neural Network simulation experiments.

This code repository includes:
  • C++ code that is compiled into a python module (gaData_module) and that reads and processes text data files
  • Python scripts that request data from gaData_module for various experiment sets and visualise it using pyCreeper

Technologies used: C++ 11, C++ Python API, Python 3, Bash scripts, Git

Apart from offering superior computational speed of C++, the gaData_module also caches data in the memory, decreasing the number of times text files need to be processed. Tests have shown that using C++ to process data makes the Python scripts execute more than 10 times faster, compared to when text files are loaded and processed by Python alone.

How it works

A schematic representation of the data analysis code It is assumed that data is organised in various folders, representing different experiments. Each folder contains data files for N runs. A python script specifies which folder to take data from and how many experimental runs to consider.

When a python script loads the gaData_module, a static instance of the module is created in the memory. This allows gaData_module to persist data, making it possible to process text files from individual runs only once and return information to various places in the python script via different function calls. In order to ensure that data from different experiments is not stored and analysed together, gaData_module also keeps track of which folder it loaded text files from and loads new data when the requested folder path changes.

{Please enable JavaScript in order to post comments}


The main purpose of pyCreeper is to wrap tens of lines of python code, required to produce graphs that look good for a publication, into functions. It takes away your need to understand various quirks of matplotlib and gives you back ready-to-use and well-documented code.

Novelty detection with robots using the Grow-When-Required Neural Network

The Grow-When-Required Neural Network implementation in simulated robot experiments using the ARGoS robot simulator.

How Coding in Python Might Be Bad For You

7 reasons why coding in Python is like writing a really bad essay and getting away with it

Designing Robot Swarms

This project looks at the challenges involved in modeling, understanding and designing of multi-robot systems.

Robustness in Foraging E-puck Swarms Through Recruitment

Swarms of five e-puck robots are used in a semi-virtual environment, facilitated by the VICON positioning system. Recruitment can make swarms more robust to noise in robot global positioning data.