Power monitoring - house wide, circuit wide or plug specific - cheap - no cloud.

[PaulCa]

Difficulty: Medium
Requires: Docker, some C&P+modify python code, databases
Cost: £50 upwards.


Objective:

Provide Grafana pretty graphs for power usage in watts over time. This may help aid your saving of electric this fall.

Secondary objective. When strategically placed, these smart monitoring plugs can be used to power down entire rooms/labs/media centres. (Control via HA)

Hardware:

Shelly EM1 - w/ 100Amp coil(s).

Shelly EM + 120A Clamp

shelly.cloud

LocalBytes Power monitor 13A(!?*) smart plug - or many of them.

Tasmota / ESPHome Smart Plug | Power Monitoring | Local Bytes

The Local Bytes Power Monitoring Smart Plug, Preflashed with Tasmota. Reminder: This comes pre-flashed with Tasmota. There will be a few steps to connect the device to your home automation system. Read More! A favourite for getting started with home automation. Automate anything that can be...

www.mylocalbytes.com

Firmware:
None - Shelly firmware is fine in local MQTT mode. LocalBytes uses Tasmota.


Software:
Docker containers for: Mosquitto, InfluxDB, Grafana and a lightweight Alpine for a python script.
Optional: HomeAssistant install (to control smart plugs)
Optional: Dockercompose

Approach.
1. Install docker on your selected host. A NAS or busy box will do. You can test it out on your windows desktop and move it later.

2. Configure and test InfluxDB docker, being careful to note where your database files live.

3. Configure and test Grafana docker, again being careful where the config DB/files lives.

4. Configure and test the Mosquitto MQTT docker.

5. Setup the hardware, as per the relevant instructions for a local MQTT server. The Shelly and Localbytes both support local MQTT. *

6. Verify you can get the hardware to send you the electrical consumption and usage data over MQTT.#

* You might want to ensure the interval rate for sending data is set to 10 seconds on Tasmota. (It's powered so no battery issues). It's a console command, "poll rate" maybe. Google.


Tying it all together:

1. A small python script which subscribes to the MQTT channel and publishs the data to InfluxDB.

2. Too many hours fiddling and tweaking to make your graphs look just way you like them in Grafana.


Sample Python script: https://gitlab.com/paulcam/home_heating/-/blob/mqtt_conversion/mqtthub/influx_mqtt_bridge.py
Sample Dockercompose: https://gitlab.com/paulcam/home_heating/-/blob/mqtt_conversion/new_docker/docker-compose.yml
Apologies these contain a bunch of other stuff in addition to the basics.

* - Treat ALL power ratings on smart devices as fictional and divide by 2. Don't run more than 6 Amps though it unless you thoroughly check it out inside first.


If there is interest I can edit this and fill out more of the details and provide links to install guides, screenshots etc.

 

[PaulCa]

A simple snap

 

[Chris Beard]

This is seriously cool. Why might someone want to do this vs. installing a smart meter please?

 

[jpaul]

Smart meter doesn't give you historic data with a small measurement resolution in minutes (30mins by default, although their are some intermediary companies who can allow you to access higher precision data from dcc cloud)

Hardware:

Shelly EM1 - w/ 100Amp coil(s).

Shelly EM + 120A Clamp

shelly.cloud

Interesting I wasn't aware of the brand :
- it's a pity their em clamp system requires mains power, as many meter cabinets in the UK are outside and have no supply, so device would need to be battery based
(https://shelly.cloud/documents/user_guide/Shelly_EM_multilanguage_v03.pdf)
- their smart dimmer looks good value if it does trailing edge for leds
( https://shop.shelly.cloud/shelly-dimmer-2-wifi-smart-home-automation#135 )

 

[PaulCa]

If your meter is outside, then your options will require a spark. There are consumer unit DIN rail style smart meters which will do similar. There are also contained units with the right regulations for UK meter installation. Note: Everything after the main meter link, is "consumer property" and you can change it via an approved spark (or unapproved if you dare).

Also, for accurate measurement you will require a voltage tap anyway. Without it, the power factor will not be accounted for and given tech in particular has poor PF you could be as much as 15% out!

Checkout "UK Meters" https://meters.co.uk/ In some instances you will get an RS485 RJ45 cable for live data, which can easily to MQTT'ized.

Anyway a few key points.

1. There is no cloud here. No utility company. No internet. The devices and the data is 100% mine and 100% local (unless I wish to access it remotely).
2. The monitoring shown is for individual plug sockets in many cases. This allows you to group related items with 4-way power bars and then meter the plug they feed from. I have 4 or 5 of these I think, so I can see individual groups usage. It also obviously shows clearly when they get switched on and off.

The former is only important if you are privacy and security aware. AND/OR you have a development mindset and having to use 3rd party commercial software and get bound into a closed eco-system would be extremely limiting.

Also of note is that this data, along with 100% of my other sensors and devices is stored in the same place and my systems have access to all of it internally. Compare this to the average smart home which has half a dozen different incompatible devices all with their own apps and cloud accounts, require Home Assistant to tame them all as best it can. No thanks.

So by having all the data in one place, I can automate things based on a wide profile of data. For example:

If it's dark outside and the lights in the office are out, and based on historical data (machine learning), the system can be fairly sure the "Non essential power" has been left on accidentally and turn it off. False positives would be annoying, but can be ironed out. That use case requires access to quite a few things, the solar panel stats to detect day/night, the office light status, the office pc status, it can even assess the heating usage by the room to determine if it's really been left vacant but with power on.

Speaking of data... I have the task (set myself) of working out....

When exactly does it become economical to switch the main gaming PC back on for the daily, which will decrease the heating usage. Trading the ~105W constant heat output at 40p a kwh, versus the estimated boiler usage when only running a 25W system instead. The extra 80W of heat, constant does make a difference.

What I really need, and is on the list, is a gas flow metering valve which I can tap and capture, then I can combine indoor temps/targets with outdoor to get delta-T and estimate much more accurate if something is saving me money or was miss guided.

 

[jpaul]

Anyway a few key points.

1. There is no cloud here. No utility company. No internet. The devices and the data is 100% mine and 100% local (unless I wish to access it remotely).
2. The monitoring shown is for individual plug sockets in many cases. This allows you to group related items with 4-way power bars and then meter the plug they feed from. I have 4 or 5 of these I think, so I can see individual groups usage. It also obviously shows clearly when they get switched on and off.

Yes indeed, I appreciate, and want, the benefit of no 3rd party involvement, just give me the raw data and a graph package+Octave.

Many UK meters do have a light pulse which can be sensed counting 1/1000ths/kwh , there are a few $$$ commercial products to do this, or homebrew NodeMCU+photodiode solutions,
so you can cut out some of inaccuracy of em sensing, and run it off a sufficiently powerful USB powerbank.

What I really need, and is on the list, is a gas flow metering valve

gas meters are open to a similar solution


Despite all the furor in the uk about energy costs consumers are currently stuck with a smart meter and 30minute monitoring on an In House Display - largely, fear of hacking means it is a closed system without an API which a smartphone app could interrogate to get dynamic , precise, data.

 

[PaulCa]

In the UK it is best to follow 1 of 2 approaches with this:

1. Stay the hell out of the meter box and do what you like.
2. Put ANYTHING in the meter box and face the wraith of the utility operator.

For example, I emailed the local gas board about fitting a pulse logger. It was lucky I did and I got a phone call from an engineer who confirmed a few things:

1. If you had installed a hobby sensor in the box, it is likely the meter reader would have flagged it as a possibly violation of "Zone 0" regulations and/or tampering and would have raised a "Cap and investigate" ticket. Which... I would have to pay for, the capping and the invetigation and uncapping by an experienced engineer. Gas can be off for 3 days in that event.
2. If I tell them, in advance, what I intend to do and they approve of it, it will be fine, it will be noted on the MRPN and the meter reader (person) will be fine with it and if they do raise a ticket it will get rejected.
3. Zone 0 is inside the meter box for the gas supply. Anything within the bounds of that box has to be zone 0 compliant. There are two categories of devices, those who are already certified and those that aren't. If they aren't it will be a longer process to get it "allowed".

He helpfully said he could supply the zone 0 magnetic pulse sensor which is a HoneyWell part available online, which is already approved. However, it requires wired input, which has other certifications about bringing any voltage in or out of the box. I didn't quite follow, but I believe it requires not only a special gland for the drill hole, but a special module which I assume limits all current and voltage spikes to remain within the Zone 0 requirements.

He also helpfully gave me the email and phone for a company who install small scale monitoring solutions for the likes of schools and small companies who have a Zone Zero approved, battery data logger which will monitor 15min intervals and publish daily. I expect it will be limiting and probably the receiver a prop. box.

At least I know it "can" be done legally and officially.

You don't want high accuracy. You are not measuring your electric to validate your bill with the utility company. You are doing it for "indicative" purposes. You want to know if you have reduced (or increased) usage by +/-5% not much more. If those 1% errors add up to £220 in a year, so what?

 

[PaulCa]

If your consumer unit is inside, you can put the coil on the input tail. That or use a DIN rail sensor.

If your consumer unit is outside in the meter box also, assuming you own the box and it's not the landlords, get a spark to fit a plug socket in it, that's what I did!

An option I forgot to put in the original post, probably are it 100% requires a spark to help you... You can buy EM3's which are 3phase monitors and connect 3 different circuits with 3 different coils. It only takes a couple of EMs to monitor all the circuits in the house. Even IF the consumer unit is outside it's likely they call present at one point that easy to access and a spark can cut them in for you.

 

[PaulCa]

Despite all the furor in the uk about energy costs consumers are currently stuck with a smart meter and 30minute monitoring on an In House Display - largely, fear of hacking means it is a closed system without an API which a smartphone app could interrogate to get dynamic , precise, data.

The smart meters don't have Wifi or Bluetooth they are often using things like LoRa, GSM, LTE and others. Lora form a mesh network.

The link to the indoor display is most like a proprietary telemetry link on the 435Mhz licensed consumer band.

The only risks for smart meters being a bad thing at this stage, is in terms of billing. A mechanical meter measures apparent power. It does not account for power factor. Thus, poor power factor gets ignored for residential customers currently (bad pun).

Industrial customers are billed very differently. They are billed on not how much they use (believe it or not!), but on they peak demand in any 15 minute period. Obviously industries can swing the grid around and generate some really high momentary demands. These demands cost a fortune to delivery so they get charged accordingly.

Also, they get billed for power factor offsets. KVar etc. Because if they use 100kW, but have a power factor of 0.9, then the power station and grid has to supply 110kW. They will get 10kW back, but they still have to supply the 110kW for it to work. Thus they bill commercial customers on their power factor reactive power.

A smart meter allows for billing to be changed from "apparent power consumption" to ... basically any form of billing, including peak demand and reactive power.

In these "environmental" times, it will be the peak demand that I see them targetting. During times of low renewables they can add a surcharge on peak demand. This would mean if you run your tumble drier during an hour they have flagged as "expensive", they slap a 10p per kwh surcharge on anything over 1kW peak demand.

The resistance they will face to that is... the good old english cup of tea. Switching on a 3kw water boiler for 1 minute will still trigger a surcharge for peak demand.

 

[jpaul]

At least I know it "can" be done legally and officially.

You don't want high accuracy. You are not measuring your electric to validate your bill with the utility company. You are doing it for "indicative" purposes. You want to know if you have reduced (or increased) usage by +/-5% not much more. If those 1% errors add up to £220 in a year, so what?

thanks , yes those magnetic monitors do seem invasive ... and as you say you don't need high accuracy gas analysis,
I only use gas for heating an IHD would enable me to do some analysis on condensing boiler settings.

If your consumer unit is inside, you can put the coil on the input tail. That or use a DIN rail sensor.

Not sure if I would need an electrician to fit a DIN rail senor (the most accurate measure)
otherwise, for total household use, do you have any accuracy concerns using coils versus led pulses, (fluctuation of voltage is the issue I've seen raised)

 

[PaulCa]

otherwise, for total household use, do you have any accuracy concerns using coils versus led pulses, (fluctuation of voltage is the issue I've seen raised)

A coil and voltage tap will give you a more accurate reading than a standard utility meter. As mentioned in terms of smart meters, the old meters are purely current driven devices. When you have a high reactive load, like your freezer (circa 50%), twice the amount of power as you use arrives and half leaves again. In a very real sense when that reactive power leaves again it "unwinds" the meter. Thus it measures only apparent power. The coil and voltage tap can do the calculation with calculus and factor in the voltage and current phase offsets to correctly determine the current direction.

So it's likely a coil+v tap will read higher than your utility meter.

I have been tracking a possible issue with my freezer the last few days, I can see the square wave of it operating far too frequently. Suspect ice stuck in the door seal, confirmed by finding it full of ice and a piece of packaging stuck in the door seal. Anyway the point of this was, I was seeing 220W of power being consumed, but I was also seeing about 120W of reactive power going back out. So, in theory, I am only being billed for 100W of that. If smart meters have their way, you will be billed for the full 220W.

As a kind of PSA, this is why you will find people burning up smart sockets and switches. They buy a switch rated for 10Amps and connect a 10Amp air conditioner to it. However the "real power" requirements of a 10Amp compressor can exceed 20Amps melting the little plugs valiant heart.