System design principles #

Before designing an IoT system that uses Actcast to infer deep learning models at edge devices, it is crucial to consider the feasibility in advance. This chapter will introduce what you should think when performing AI tasks that perform image processing in an IoT system.

Characteristics of an image processing system using deep learning #

Points:
Tasks that could not be realized in the past can now be automated
It is based on heuristic processing, and accuracy cannot be guaranteed.
The accuracy of the task depends on the quality of the data used for learning and should be improved while operating an IoT system.
Accuracy may deteriorate due to changes in the environment (amount of light, imaging angle, properties of processing targets)

With the development of deep learning, it has become possible to perform image processing tasks with higher accuracy than conventional methods. However, it still relies on heuristic methods, so recognition and identification accuracy is not 100%. Therefore, functions that depend on this task must be error-tolerant. Machine learning tasks are suitable for counting the number of store visitors, which was not previously counted for marketing, although it is risky to perform personal identification at the security gate only by image recognition.

Image processing systems that use machine learning are usually created by supervised learning. Therefore, the accuracy of the task depends on the quality of the data used for training. You may not always have the right data for the job you want to do when you start running your system, and you will need to collect data and improve accuracy while running your system using machine learning.

Since the accuracy of an image processing system using machine learning depends on the learning data, the accuracy may decrease if the object’s properties in the real world change. For example, when considering a system that recognizes a person’s face, people began to wear masks rapidly after 2020. To keep up with such changes, it is necessary to continually check the accuracy of the task, and if the accuracy drops, continue to improve. If that cannot be improved in-house, it is necessary to find a vendor that can develop and operate it and build a long-term relationship.

When performing machine learning tasks on people to collect and process information, they may be regulated by laws and regulations to protect personal information in each country.

Trade-off between accuracy and complexity in deep learning models #

If you write your own app, you have to keep in mind that there is a trade-off between accuracy and execution time when deciding which deep learning model to use.

In general, models with a large amount of calculation tend to improve accuracy, and lightweight models with a reduced amount of calculation may have lower accuracy. Lightweight models used in edge terminals include MobileNetV3, for example, when classifying images, but there is a trade-off between model accuracy and execution time, and it should be appropriately used according to system requirements such as response time. ..

For example, if the response speed is only 1 second, you can use a huge model with reasonable accuracy. On the other hand, if the required response speed is 0.1 seconds, you should use a lighter model with less accuracy.

Characteristics of IoT system #

Points:
Many cheap devices connect to the internet
Need to tolerate internet instability
Need to allow the device to break

An IoT system is a system that connects a large number of inexpensive devices to the internet to process information. When thinking about IoT systems, it should be considered that the internet is regularly experiencing minor glitches. For example, small momentary interruptions are always occurring somewhere. It is not suitable for work that has a significant impact if it is stopped even temporarily.

It also features a large number of inexpensive devices participating in the system. Therefore, it is necessary to define a business flow that assumes that the device will fail.

See Device Procurement and Selection for device procurement selection.

Characteristics of the Actcast app #

Points:
Actcast apps are constrained Linux applications
There are restrictions on utilizing high-speed technology
The Actcast app is constrained because it scales as an IoT system

The Actcast application is a constrained Linux application. It is assumed that the device is always connected to the internet.

Actcast’s online compiler makes deep learning models run fast on edge terminals. To take advantage of Actcast’s acceleration technology, development vendors need their machine learning models to follow a format that describes general-purpose deep learning models called computational graphs. See Model Conversion and its limitations for details.

The typical constraints of an Actcast application are:

Communication from the device to the server (Actcast) is possible, vice versa
Data cannot be stored locally
Not all peripherals can be used
- Cannot use peripherals that require rewriting the OS image
  - It is necessary to install a particular driver, etc.
- peripherals connected via Bluetooth are not available

Confirmation of installation environment #

Points:
Preliminary survey of the surrounding environment is indispensable for stable operation
How to connect to the network
Whether to get a steady power supply
Whether to make cameras get the desired angle of view
Whether to do installation and maintenance smoothly

When considering the installation location, it is necessary to confirm whether the IoT device can be operated stably. The following items are required to use Actcast.

Stable network environment
Stable power supply

The method using a wired connection is the most stable.

When using a wireless connection, you need to be aware of the nature of the wireless. There are two available bands, 2.4 GHz and 5 GHz, but each has its advantages and disadvantages. It is said that 2.4 GHz is more resistant to interference from walls, but it interferes with microwave ovens. In an environment such as a multi-tenant building, the effects of 2.4 GHz crosstalk are serious. Regardless of which band you use, the transfer speed will decrease as the distance from the router increases, so you may not be able to achieve the desired performance even if you try to connect. Also, wireless connection may not be available for all Raspberry Pi products. You should avoid access points such as Free Wi-Fi, which is open to the general public, whose stability may be compromised due to factors other than your system.

You can also use pay-as-you-go SIM communication. You should estimate the amount of communication in advance in consideration of the communication frequency of the app.

When installing a device with a camera, you need to check in advance that it can shoot from the desired angle. The expected accuracy may not be achieved depending on the conditions such as angle and amount of light.

In order to carry out installation work and maintenance smoothly, it is necessary to confirm in advance whether the installation environment can be controlled. For example, when installing in a rental office or franchise store, the time, schedule, and installation position of the installation work may be restricted by the owner’s intention. Also, it is necessary to define the failure response flow in advance and seek subcontractor’s cooperation.

Legal request #

When collecting information that can identify an individual (personal information) using a sensing device such as a camera, it is subject to legal restrictions.

Japan #

Practical notices using camera pictures:

カメラ画像利活用ガイドブック