Can AI Build Your Smart Home? ChatGPT vs. Gemini

Introduction

Generative AI is everywhere. People are using AI to write essays, generate images, create videos, have a conversation, and so much more. Which got me wondering…can AI help me build my smart home?

I thought it’d be fun to test the ability of OpenAI’s ChatGPT and Google’s Gemini to write smart home automations. We can compare the results between the two, and see if they actually work.

Main Points

I’m going to give five of the exact same prompt to ChatGPT 3.5 and Gemini, and see if either is good enough to automate my smart home. I’m using Home Assistant as my smart home hub, so to really test these AI models, I’m going to ask them to give me the answer by writing the YAML configuration.

The prompts I give them will get increasingly more complex as we go, and I’ll show you how quickly each model responds to my prompt in real time. For fun, I’lll also give an arbitrary score to both models to see which one is best at writing smart home automations. Watch the YouTube video to see how ChatGPT and Gemini handle my automation prompts.

1) Write the yaml code for a smart home automation to close the garage door if I forgot to close it and nobody is home.

ChatGPT

The first thing I notice with ChatGPT is just how quick it is. The response is immediate. Looking at the response, I like how it includes instructions, like remembering to use the correct entity name, and where you can customize to suit your preferences, like how long to wait before triggering the automation.

On the condition, I believe you could also use the cover entity for your garage door, and check if the state is open. Of course the trigger does not consider if there are multiple device trackers to consider, or how to account for those in the home without a device. But overall, this did a decent job.

Gemini

Here we see that Gemini takes longer to respond, roughly 5 seconds compared with 1 second or less with ChatGPT. The next thing I notice is how the two have taken a different approach to the trigger. ChatGPT had the automation trigger based on the location of a device tracker, whereas Gemini triggers based on the state of the garage door or presence detection using a template sensor.

Likewise, the condition from Gemini seeks to confirm that the garage door is open, and that no one is home after a time interval. Gemini also includes more instruction to adapt to your devices and preferences, not only with in-line guidance, but quite a bit of notes beneath the YAML configuration. It even has a warning safety when closing the garage door.

While the template sensor would need additional knowledge to adapt to your smart home, I like that Gemini considered how presence detection is more than just the location of a single device tracker.

Scoring

I’ll give one point to each for a relatively usable automation, plus point to ChatGPT for being much faster, and one point to Gemini for a more thoughtful response.

ChatGPT: 2

Gemini: 2

2) Write the yaml code for a smart home automation to turn the kitchen lights on when someone enters the room and it’s between 6:45 am and 11:00 pm, and to turn the kitchen lights off if no presence is detected in the kitchen for 10 minutes.

ChatGPT

ChatGPT crushes this request with an instant response. But it doesn’t really peel the onion on presence detection, using the same generic binary sensor for presence on and presence off.

I like to use passive infrared motion sensors for turning the lights on, and then mmWave to understand true presence detection and to turn the lights off. Using passive infrared motion or mmWave for both lights on and lights off has downsides, so that kind of nuance is not reflected here. But ChatGPT does give two workable, simple automations to turn the lights on and off at the requested times and intervals.

Gemini

Gemini took its time, with about six seconds to respond. I also noticed it responded in phases, vs. giving everything all at once. It gave me everything up until light turn on service, paused briefly, and then continued, and stopped with a half-written entity ID for the light at the very end, before resuming and finishing that out, which was a bit curious.

Gemini also took an unexpected approach with the configuration. The automation is triggered by motion or the time of day, with a condition to only turn on the lights before or after sunset, something I did not request or want.

Like ChatGPT, it uses the same sensor for lights on and off, but specific that it is a motion sensor. I also think that “home” and “not_home” are atypical states for a motion sensor.

Scoring

I’m gonna give ChatGPT one point for speed and overall accuracy, and no points to Gemini for adding conditions that I did not want, and ignoring my stated condition on the time interval.

ChatGPT: 3

Gemini: 2

3) Write the yaml code for a smart home automation to remind me to take the trash out every Thursday night at 8:00 pm, and continue to remind me every 15 minutes until I dismiss the reminder by scanning an NFC tag with my iPhone.

ChatGPT

Once again, ChatGPT responded within one second or less. We can see it provided two automations: one for the weekly reminder, and one for dismissing the reminder by scanning an NFC tag. I like how the notify service is specific to iPhone as I requested, and includes a title and message.

I’m a little confused by its second automation for dismissing the reminder. It appears to say, if the NFC tag is scanned, check to see if the first automation for the main reminder is on, and if it is, then turn it off, but then wait 15 minutes and turn it back on again. However, I want it to dismiss the reminder if I scan the NFC tag.

Gemini

Gemini took about six seconds to respond this time, so slightly longer than the first scenario, and a lot slower than ChatGPT. Once again we see the different approach taken by Gemini vs. ChatGPT. The notify service is not specific to my iPhone as requested, and the message does not include a title, but these are harmless.

Gemini uses a loop command for the reminder every 15 minutes, which honestly I have never seen before, versus something like repeat or sequence. Gemini also includes the configuration template for the NFC tag, in addition to the two automations: one for the reminder, and one for dismissing it.

The actual configuration of both automations is a bit different, but you can achieve the same automation outcome using more than one approach.

Scoring

I’ll give one point to ChatGPT for being super fast and providing a better notify service configuration. Gemini gets one point for appearing to better understand my automation intent.

ChatGPT: 4

Gemini: 3

4) Write the yaml code for a smart home automation to display an image of the front doorbell camera on my Sony TV when a person is detected by the front doorbell camera, and send a push notification to my iPhone with the image from the front doorbell camera, and announce on my Echo device that someone is at the front door but only if it’s not between 1:00 pm and 3:00 pm.

ChatGPT

ChatGPT of course responded right away. The YAML configuration is decent, but a couple things that I notice. The binary sensor shown for detecting motion is not necessarily specific to person detection. The entire automation has the condition to only run before 1:00 pm or after 3:00 pm, but I only wanted that condition for the action to make an announcement on the Echo.

I don’t think the notification to the TV would work as written without a supported integration. And this time, ChatGPT does not provide any additional explanation or guidance text either in-line or at the bottom that would help you adapt to your smart home devices.

Gemini

This was the slowest reply yet for Gemini, at about seven seconds. I like how Gemini uses a person detection sensor to trigger the automation, but it failed on the conditions, doing the opposite of what I requested. The entire automation would run only between 1:00 pm and 3:00 pm, but my request was to only announce on the Echo device that someone is at the front door if it’s before 1:00 pm or after 3:00 pm (which is when my kids are napping). I also found Gemini’s image URL less clear than ChatGPT.

Scoring

Both models get zero points in this round for failing to provide the condition as intended. Of course, I realize that may come down to the way I worded the request.

ChatGPT: 4

Gemini: 3

5) Write the yaml code for a smart home automation to send me an actionable push notification to my iPhone asking if i’d like to arm the alarm with options to select either arm away mode or ignore, but only send me the actionable push notification if the alarm is disarmed and nobody is home.

ChatGPT

While it replied instantly, ChatGPT bombed this one. The automation it proposed is triggered when the alarm is set to away, which is the opposite of my request. If it’s already in away mode and nobody is home, why do I need a push notification asking if I’d like to arm the alarm? ChatGPT also does not consider the requirements to build an actionable push notification. In my experience, you need multiple automations to make this work.

Gemini

Ironically, this was the fastest that Gemini responded — about three seconds — when it was meant to be the most complex. It uses two triggers — one for alarm state and one for person state — but only specifies what state the alarm should be in. It then has a condition for the person state, and not the alarm state. I found this approach a bit curious. I believe this was the first time that Gemini included a title along with the message — not sure why.

What was cool is that Gemini understood additional automations are needed to respond to the action buttons in the push notification — something ChatGPT missed. I wanted Gemini to give me those additional automations, so I sent a follow-up reply requesting them, which it then provided in about three seconds.

Scoring

ChatGPT gets no points for the second round in a row, and Gemini gets one point for understanding additional automations were needed for an actionable push notification.

ChatGPT: 4

Gemini: 4

Final Thoughts

So the final score is a tie: ChatGPT 4, and Gemini 4. What are my takeaways, and is AI ready to build your smart home?

First, neither AI model is perfect at writing YAML configuration for smart home automations. Sometimes they get it right, and sometimes they are dead wrong. If you’re working on an automation in Home Assistant and aren’t sure how to proceed, ChatGPT or Gemini may help point you in the right direction, but take their proposals with a pinch of salt.

Second, ChatGPT is hands down faster than Gemini. Depending on the scenario, ChatGPT responded up to seven times faster. While Gemini’s responses generally slowed a bit as the request got more complicated, ChatGPT always responded instantly regardless of request complexity. I should point out that ChatGPT and Gemini take different approaches to displaying responses: ChatGPT responds immediately by writing each line in real time, whereas Gemini sends replies in batches with numerous lines at a time.

Third, it’s a reminder that there isn’t only one way to write an automation using YAML configuration. I don’t think AI is ready to fully automate your smart home, but it can definitely help if you’re working on an automation idea and looking for guidance — just know that the guidance may be…misguided.

Watch on YouTube

Featured Tech

Smart Plug - TP-Link Kasa Smart Plug Mini with Energy Monitoring: https://amzn.to/3O0G9PX
Vibration Sensor - Aeotec SmartThings Multipurpose Sensor: https://amzn.to/47BIIix
Zigbee Coordinator - SONOFF Zigbee 3.0 USB Dongle Plus Gateway: https://amzn.to/3vqQfmP
Thermostat - Honeywell Home T5 WiFi Smart Thermostat: https://amzn.to/3Sz4XBf
Philips Hue Smart 60W A19 LED Bulb - White and Color Ambiance Color - 1 Pack: https://amzn.to/3vHIsBf
Philips Hue Smart 60W A19 LED Bulb - White Ambiance Warm-to-Cool White Light - 2 Pack: https://amzn.to/427c8nr
Philips Hue Smart 75W A19 LED Bulb - White and Color Ambiance Color-Changing Light - 2 Pack: https://amzn.to/3Sp2QQD
Philips Hue Smart 75W A19 LED Bulb - White Ambiance Warm-to-Cool White Light - 2 Pack: https://amzn.to/3vH5sAy
Philips Hue Smart 100W A21 LED Bulb - White and Color Ambiance Color - 1 Pack: https://amzn.to/3OaHwvH
Philips Hue Smart 100W A21 LED Bulb - White Ambiance Warm-to-Cool White Light - 1 Pack: https://amzn.to/3OdA4jv
Philips Hue Smart 40W B39 Candle-Shaped LED Bulb - White and Color Ambiance Color-Changing Light - 1 Pack: https://amzn.to/47LhfLs
Philips Hue Smart 40W B39 Candle-Shaped LED Bulb - White Ambiance Warm-to-Cool White Light - 1 Pack: https://amzn.to/3U4AV9V