Lab 5

Contents

• Introduction
• TinyML- Part 1
• Part 2
• Follow Up and LLM Integration
• Further Experiments

Introduction

You have been working with Micropython so far to develop various sensor interfacing and signal processing aspects. You can push along this frontier and for simpler ML models e.g. decision trees etc. You can still implement everything using Micropython. This can be done using the library called EMLearn. We have a full lab developed using EMlearn which you can download and follow the instructions in folder here: Download . I would recommend skipping this for now and doing this in your own time.

However, when it comes to complex ML models e.g. CNNs or LSTM which you need for your project, memory management and processing power becomes key. You have to buffer sensor data while run ML pipeline on the device. Therefore, Micropython often becomes limiting factor and fastest way to deal with this is writing directly in Arduino C. On the flip side, there are far more pre-built libraries in C which can simplify the job. To that end, in this semester our remaining two labs will be in C.

Before we begin anything, we need to do few things:

1. Bring your devices to lab instructors they will wipe off micropython firmware.
2. Collect SD card and card readers as you need these.

TinyML- Part 1

The first part of this lab is designed to be quick and informal hack style approach. I believe this is needed to enable you to transition from structured approach to look for the tutorials yourself and implement it. This will be useful skill for your projects. Also there is no point for me to rewrite already written tutorials. Therefore, for first part I suggest you follow the following tutorial from Seeed on TinyML.

Start with the Image Classification Tutorial hereby:Image Classification

Part 2

Once done with the Image Classification Tutorial, continue with the Keyword Spotting

Follow Up and LLM Integration

If you are done with the both of these then try following it up with the LLM integration with Xiao here with an LLM . Notice you do not have screens so you need to be creative to display output. Think about transmitting it over BLE. You cannot use your micropython BLE code but there are plentty of pre-built libraries in C. Also the HTML webpage showing Web Bluetooth output should still work with minor tweaks.

Further Experiments

Searchup WiFiManager, the library allows managing WiFi connectivity and also provide support for MQTT. Check if you can publish the images as base64 encoded string from camera over MQTT.