Chapter 4

Working with Large Data Sets

In this chapter, we are going to take a step forward in our data science experimentation by significantly increasing the volume of data that we collect. So far we have worked with a dozen data points, and that was adequate for our needs. But sometimes correlations (and other interesting features of data) are subtle and hard to find. The larger our data set the more likely it is that the answers we are looking for lie within it.

We use microPython in this chapter to take advantage of the built-in file system.  The code used is listed below:

Section 4.2:

Download pre-compiled hex file (in a .zip folder)

with open('hello.txt', 'w') as my_file:
    my_file.write("Hello, World!")

Section 4.7:

Download pre-compiled hex file (in a .zip folder)

from microbit import *
from SW01 import SW01

import os
import sys

SW01 = SW01()
d = 0

try:
    d = os.size("we.csv")
except:
    pass

if (d > 0):
    display.show("*")
    sys.exit()

open("we.csv",'w')

while True:
    try:
        with open("we.csv") as a:
            cont = a.read()
            cont = cont + str(SW01.getTempC()) + "," + str(SW01.getHumidity()) + "\n"
    except:
        display.show(".")
        sys.exit()

    with open("we.csv", 'w') as b:
        b.write(cont)

Section 4.8:

Download pre-compiled hex file (in a .zip folder)

from microbit import *
from SW01 import SW01

import os
import sys

SW01 = SW01()
d = 0

try:
    d = os.size("we.csv")
except:
    pass

if (d > 0):
    display.show("*")
    sys.exit()

bp = False

while True:
    if button_a.was_pressed():
        a = open("we.csv", "w")
        bp = True
    if (bp == True):    
        try:
            with open("we.csv") as a:
                cont = a.read()
                cont = cont + str(SW01.getTempC()) + "," + str(SW01.getHumidity()) + "\n"
        except:
            pass
            display.show(".")
        with open("we.csv", 'w') as b:
            b.write(cont)

        sleep(1440000)      # Every [x] minutes