AWS IoT connected SmartHome

Our Smart Home innovation project has won the international KNX award 2018 in category 'Special'. The project combines AWS IoT Cloud Services, Machine Learning and AR Augmented Reality with the KNX communications protocol for building automation.

KNX fits perfectly into the modern world of IoT, cloud computing, machine learning and augmented reality, even if it has been around for decades. In this SmartHome we have connected the KNX bus to a local MQTT hub and also a gateway to communicate with the AWS (Amazon Web Services) cloud IoT Hub. KNX and MQTT messages have much in common, combining both in a bi-directional way will open up endless possibilities.

Once the KNX bus can communicate with AWS over MQTT and vice versa using the interface software written by WebGate, creating custom Alexa Skills to control your KNX connected devices is easy. Using your voice and Alexa you can control lighting, ventilation, roof windows, blinds, multimedia system, TV, kitchen fan and so on. A local node.js gateway will forward the intents received from the AWS cloud to the KNX bus and receive commands from the cloud.

In this SmartHome you rarely have to open a window, thanks to automated air ventilation and “Minergie ®” low energy standard of the building hull. However some time ago, someone left a window open in an unattended room and just forgot about it. It took some time to notice this. We’ve been looking for a solution on how to avoid such happenings in the future, but the windows don’t have sensors to report if they are open or closed. So we came up with the idea to use machine learning to detect such an event in the future. Machine learning is excellent in detecting patterns, so we captured tons of data from the KNX bus feeded it into AWS Machine Learning to create a ML model. Instead of using fixed rule sets and myriad of dependencies, where you have to think about each and every possible situation, machine learning can find patterns in your data by looking at the big picture and decide if a certain sensor reading makes sense in conjunction with data from other sensors. Now if you leave a window open for a certain time, the room temperature, humidity and even VOC values will change and the machine learning algorithm will detect this anomaly and report it to the residents to check a certain room. It can not only detect an open window, but also alerts if a component has failed, e.g. the ventilation stops working.

In each room there’s a small in-wall speaker for speech output, powered by AWS Polly. The house can announce various kinds of events of KNX connected devices in a friendly german female voice e.g. when a letter has been delivered to the letter box, when the garage door is still open after a longer period and will be closed automatically, when the roof windows open or close, or even warn you when the KNX connected weather station has triggered a wind alarm, so you immediately know why the blinds are opening in such a case.

The KNX connected blinds of the house are controlled by a cloud based custom logic based on NodeJS. Sensor readings are sent to the cloud and the blinds are controlled not just based on room temperature and energy demand, but also the angle of the blinds are precisely adjusted using GPS coordinates, sun azimuth and elevation, to open up the blinds as much as possible, but still prevent direct sunbeams to enter the room. The ventilation is also controlled by the AWS cloud, based on VOC sensors and smoke sensors it can detect when the inbound air is polluted by smoke of the neighbours chimney or if there’s a strong odor by farmers fertilizing the acres.

Connecting KNX to MQTT literally allows every device to talk to any other device. The blinds in the living room know if the TV is turned on, and adjust the blinds accordingly depending on sun position and intensity. On an incoming phone call, TV and audio will be muted, the VoIP phones ring. When the door bell rings, a picture of the visitor is not only displayed on all info panels, it also submits the visitors image to the AWS Rekognition image analysis service to detect and report the full name of the visitor. The visitors name will be announced using AWS Polly Speech Synthesis, when the visitor has been previously registered in the database.

Using the Microsoft HoloLens augmented reality glasses you can take it even further and for instance eliminate traditional wall switches or enrich your environment with augmented objects. We created a HoloLens app, also connected to the KNX bus using MQTT. In this app you can place virtual switches on the walls wherever you want, or display sensor readings and device status. There is also a small virtual clone of the house which you can expand using a fingertip and see realtime KNX sensor readings or switch lights. The HoloLens developer preview hardware is currently still a bulky device, but in the future devices will get smaller and replace traditional screens and smartphones. Possibilities are truly endless and the KNX bus is a valuable component of a modern SmartHome. This is just an excerpt of all the implemented features in this SmartHome.