MAX78000FTHR- Edge AI

Updated content on November 17th - Edge AI applications of MAX78000FTHR

In the previous content, I used the MAX78000FTHR development board + the expansion board in this project to implement the function of a simple camera that can capture pictures and save them to an SD card.

In this update, I will use the MAX78000FTHR development board + the expansion board in this project (hereinafter referred to as the MAX78000FTHR expansion) to implement the classification and recognition of three car logos, and explain the implementation process.

Demonstration of the effects of classification and recognition of three types of car logos

The three car brands are Wuling, Volkswagen and Mazda. The following three pictures are the recognition phenomena of the logos of these three brands respectively.

Wuling vehicle logo recognition.

Volkswagen car logo recognition.

Mazda car logo recognition.

Training implementation process

The model training process is divided into the following steps:

Model training-> model quantification-> model evaluation-> model conversion

The following describes these four steps:

1. Three-category training

Open the train.py file in the training project. This training file requires input parameters. The following uses cat and dog recognition as an example for explanation.

--epochs 250 --optimizer Adam --lr 0.001 --wd 0 --deterministic --compress policies/schedule-catsdogs.yaml --model ai85cdnet --dataset vehicle_logo --confusion --param-hist --embedding --device MAX78000

Copy it to the running parameters of train.py, as shown below.

Then click to run the train.py file. It will automatically load the parameters we just entered and start training.

After the training is completed, the files required for quantification in the next step will be generated in the logs folder, as shown in the figure below.

2. Quantification

Copy the best.pth.tar file generated in the previous step to the trained folder under the quantitative project.

Open the quantize.py file and configure the running parameters for the file, as shown in the figure below.

The operating parameters ：trained/best.pth.tar trained/best-q.pth.tar --device MAX78000 -v

After the configuration is completed, run the quantize.py file. The best-q.pth.tar generated in the trained file after the operation is completed is the quantized model file.

3. Model evaluation

After obtaining the quantized model file, we switch back to the training project to evaluate the model.

The file used for model evaluation is the same file used for model training, but the running parameters are different. Here we modify the running parameters of train.py to evaluate the model.

The evaluation input parameters are as follows:

--model ai85cdnet --dataset vehicle_logo --confusion --evaluate --exp-load-weights-from ../ai8x-synthesis/trained/best-q.pth.tar -8 --device MAX78000 

Run the train.py file and the results are as follows:

The effect is okay.

4. Model conversion

Model conversion is to convert the trained model into a C file so that we can conveniently deploy the model.

Switch to the quantitative project again. This time we need to use the ai8xize.py file. Similarly, we need to configure the running parameters for the file to complete the model conversion. The running parameters of the file are as follows:

--verbose --test-dir demos --prefix ai85-vehicle_logo_3 --checkpoint-file trained/best-q.pth.tar --config-file networks/vehicle_logo-hwc.yaml --fifo --device MAX78000 --softmax

After running, the development board project files we need will be generated in the demos folder under the quantitative project, as shown in the following figure:

The project file can be opened and edited using VScode.

Model deployment

In the previous step, I implemented the training of the model and finally obtained the project file of the chip, but what was generated here was only a sample. The data input to the model was a sample generated by the training project, and the camera cannot collect it. The obtained image is sent to the model, so in this step we deploy the model to an available project.

1. Model migration

First open the project folder converted from the model and copy the cnn.c/h, softmax.c and weights.h files therein.

Then paste it into the cats-dogs_demo routine (this routine can send the image collected by the camera to the model). When pasting, the window in the picture below will pop up. Select the option "Replace the file in the target"

In this way, we will migrate the trained model to the original cat and dog classification project.

2. Program modification

First modify the screen initialization function to adapt to the screen of the expansion board. Place the following code in the screen initialization position of the main function, as shown in the figure below.

Then modify the classification definition in the program, that is, the box selection in the picture below.

This definition is used to distinguish the content of categories. Change the array to the content shown in the figure below. After modification, it is shown in the figure below.

Add the expansion board key code as shown below.

After this initialization, we can get the high and low levels of the pin through the MXC_GPIO_InGet() function to know whether the button is pressed. Since when the button is pressed, the pin level will be pulled low, so use MXC_GPIO_InGet(gpio_set .port, gpio_set.mask) != 0 replaces all !PB_Get(0) code in the project.

Then modify the camera image refresh function part (note that this function is in the main function) to the following function.

The location is shown below.

The reason for this modification is that the source program can not always browse the content taken by the video head, but can only observe the shot picture when shooting, which is not conducive to the shooting, and this modification can always observe the camera picture, so as to capture their own satisfactory picture.

But this requires adding a delay function after the next key press judgment to prevent accidental touches, as shown in the figure below.

However, due to the placement of the screen on my expansion board and the development board, the angle of the camera image will be 90 degrees perpendicular to the angle at which people view the screen, so we need to modify the camera image display function.

We will inside the capture_process_camera function MXC_TFT_ShowImageCameraRGB565(TFT_X_START, TFT_Y_START + row, data565, w, 1); Replace with MXC_TFT_ShowImageCameraRGB565(TFT_X_START + h - row, TFT_Y_START, data565, 1, h); You can rotate the camera screen around.

The principle of doing this is basically that the camera data we obtain is displayed line by line. The display effect of the original function is rotated 90 degrees to the left, so the data of the first line of the captured image should be The data in the rightmost column of the normal image, so after we obtain the image data, we do not display it row by row, but display it column by column. At the same time, starting from the rightmost side, the processed image looks like Normal effect.

The location of this line of code is shown below.

For the three-category identification code, please see the attachment WuLing-VW-Mazda.7z

The above is the entire content of the program modification. For the effect display, please see the effect display section of three types of car logo classification recognition.

The quantitative project is too big and has been put on the network disk. A network disk link will be sent out in the comment area later.

The above is all the content of this update of the edge AI application of MAX78000FTHR.

Content before November 17th

Project analysis

This project is a MAX78000FTHR expansion board that can capture and store images to an SD card.

This project can easily use the MAX78000FTHR to collect images and obtain the actual pictures captured by the onboard camera, thus greatly improving the efficiency of data set acquisition. Using this kind of image for model training of MAX78000 can improve the accuracy of camera recognition.

The picture below shows the hardware structure diagram.

The following figure shows the software structure diagram.

Schematic design instructions

This project uses TP5400 for power management

When powered by an external power supply, it can charge a 3.7V lithium battery and output a 5V voltage.

When the external power supply is not powered, the 3.7V voltage can be boosted to 5V.

Thus realizing charging and boosting two-in-one.

The screen display part uses a 2.4-inch screen, the driver is ILI9341, you can directly use the screen driver officially provided by MAX78000, the hardware connection is as follows.

For the key operation part, because the keys on the MAX78000FTHR are too mini and inconvenient to operate, the keys are reconnected on this expansion board to realize the key operation. The schematic diagram is as follows.

The connection between MAX78000FTHR and other parts is shown in the figure below.

PCB design instructions

In the PCD design part, this project uses color silk screen printing, which is very beautiful. The 3D preview is shown below.

The actual picture is shown below.

Note: If you do not want to use color silk screen to print the PCB board, you can directly delete the color silk screen image in the PCB, which will not affect the circuit operation, or directly use the ordinary silk screen Gerber file in the attachment to print the board.

Physical display instructions

Front view

Reverse view

Bug update on November 10th

There is a bug in the camera shooting program, that is, after pressing the shooting button, the camera shooting screen when the button is pressed will be displayed on the screen, but when the save button is pressed, the camera shooting screen when the save button is pressed is saved. , which is problematic.

Solution:

The picture when the shooting button is pressed is saved when the button is pressed, saved to the SD card when the save button is pressed, and deleted when the discard button is pressed. The key code part is as follows.

The project in the attachment has been updated.