Howto control a Xiaomi Robot Vacuum without app using Valetudo

I’ve tinkered before with my Xiaomi Robot Vacuum but returned to the official Xiaomi app since the existing solutions felt uncomfortable. I even worked on adding Mac support for the dustcloud software but stopped using the rooted firmware.

A few days ago I’ve read about Valetudo. Valetudo is a web interface to the Xiaomi robot being self hosted on the robot. It allows easy extraction of the necessary control token and stops the robot from reporting cleaning and location data to Xiaomi. There’s also support for MQTT so that you can integrate it into existing home automation systems.

I followed the instructions on creating a rooted firmware and found a few problems and want to share my solution:

  • The firmware builder creates a firmware package along with SSH keys supplied during the build process. I could not login using those SSH keys and required the SSH key directly from the ~/.ssh folder of the user.
  • Flashing inside a VirtualBox Ubuntu VM doesn’t work, even when you use a bridged network interface. You maybe able to request the device token but the flash command always fail.
  • Flashing the robot may fail, if it isn’t completely reset to its default. You can reset the robot to factory default by pressing the home and reset button until you hear the chinese voice.
  • You should flash the robot while it is inside its charging station.
  • If you’re using a Mac, you can install python3 and the required python packages. This will allow you to flash the firmware directly from your mac.
  • Keep your machine close to the robot during the flashing process, because it might otherwise timeout.
  • Since I’m using a chinese version of the robot, I only hear the chinese voice. In this case you’ll need to convert the robot to a european version following these instructions. Once the robot is rebooted you’ll hear the english translation and can verify this from the Valetudo interface.

Now you’re ready to use Valetudo. I’ve added a link to the Valetudo homepage on my smartphone. It replaces now the Xiaomi app while it still provides access to the cleaning map, the maintenance hours for replacing parts as well as automated clean up plans. All in all its a really nice piece of software!

Disable macOS Catalina update notification in Mojave

macOS Catalina was released and is ready to install. If you’re using the previous macOS version called Mojave, you’ll get a notification badge on the system settings.

This little red notification badge is really annoying.

The following two commands were taken from the Apple support forum:

sudo softwareupdate --ignore "macOS Catalina"

If you want to install Catalina via the software update, you can reset the ignored updates with this command:

sudo softwareupdate --reset-ignored

This will hide successfully the Catalina update from the list of available updates in Software Update. However, it won’t remove the notification badge.

But fortunately you can even disable the badge by using these commands:

defaults write com.apple.systempreferences AttentionPrefBundleIDs 0
killall Dock

This will hide the badge until the next time you’ll scan for available software updates.

Monitor Fritz!Box connection statistics with Grafana, InfluxDB and Raspberry Pi

I’ve recently stumbled over an article in the german magazine C’T about visualisations of your Fritz!Box’s connection. The solution looked quite boring and outdated, since it used MRTG for the graph creation.

I’ve started searching for a better solution using Grafana, InfluxDB and my Raspberry Pi and found this great blog post. I’ve already explained how to install Grafana and InfluxDB in this post, so I’ll concentrate on the Fritz!Box related parts:

Start with the installation of fritzcollectd. It is a plugin for collectd.

sudo apt-get install -y python-pip
sudo apt-get install -y libxml2-dev libxslt1-dev
sudo pip install fritzcollectd

Now create a user account in the Fritz!Box for collectd. Go to System, Fritz!Box-user and create a new user with password, who has access from internet disabled. The important part is to enable „Fritz!Box settings“.

Additionally make sure that your Fritz!Box is configured to support connection queries using UPnP. You can configure this under „Home Network > Network > Networksettings“. Select „Allow access for applications“ as well as „Statusinformation using UPnP“.

Next part is the installation and configuration of collectd:

sudo apt-get install -y collectd
sudo nano /etc/collectd/collectd.conf

Enable the python and network plugins by removing the hashtag

LoadPlugin python
[...]
LoadPlugin network

Scroll down till you’ll see the plugin configuration and configure the port and IP for collectd

<Plugin network>
    Server "127.0.0.1" "25826"
</Plugin>

Enable the python plugin and configure the module with the username and password of the user you’ve created. Make also sure to use the right address.

<Plugin python>
    Import "fritzcollectd"

    <Module fritzcollectd>
        Address "fritz.box"
        Port 49000
        User "user"
        Password "password"
        Hostname "FritzBox"
        Instance "1"
        Verbose "False"
    </Module>
</Plugin>

Since you’ve already got a running InfluxDB, you’ll just need to enable collectd as data source:

sudo nano /etc/influxdb/influxdb.conf

Search for the [collectd] part and replace it with

[[collectd]]
  enabled = true
  bind-address = "127.0.0.1:25826"
  database = "collectd"
  typesdb = "/usr/share/collectd/types.db"

Reboot collectd and influx to activate the changes made

sudo systemctl restart collectd
sudo systemctl restart influxdb

Login to your grafana installation and configure a new datasource. Make sure to set the collectd database. If you’re using credentials for the InfluxDB, you can add them now. If you’re not using authentication you can disable the „With credentials“ checkbox.

Check if your configuration is working by clicking on „Save & Test“.

If everything worked, you can proceed to importing the Fritz!Box Dashboard from the Grafana.com dashboard. The ID is 713. Make sure to select the right InfluxDB during the import setup.

After clicking on import, you’ll should be able to see your new Dashboard. It might take a few minutes/hours until you’ve gathered enough data to properly display graphs.

Be aware though that if you start gathering this much data you’ll might end up with „insufficient memory“ errors. You’ll might want to tweak your InfluxDB settings accordingly.

Crashing influxdb on Raspberry Pi 3+ because insufficient memory

A few days ago I’ve noticed that my influxdb installation wasn’t working properly. The server was crashing constantly.

I’ve checked the logs using

sudo journalctl -u influxdb -b

and found this

May 12 23:12:18 pi3plus influxd[30173]: ts=2019-05-12T21:12:18.440902Z lvl=info msg="Opened file" log_id=0FNU47~W000 engine=tsm1 service=filestore path=/mnt/databases/influxdb/data/_internal/monitor/342/000000020-000000002.tsm id=0 duration=14
May 12 23:12:18 pi3plus influxd[30173]: runtime: out of memory: cannot allocate 2121015296-byte block (16056320 in use)
May 12 23:12:18 pi3plus influxd[30173]: fatal error: out of memory
May 12 23:12:18 pi3plus influxd[30173]: runtime stack:
May 12 23:12:18 pi3plus influxd[30173]: runtime.throw(0xbc70be, 0xd)
May 12 23:12:18 pi3plus influxd[30173]:         /usr/local/go/src/runtime/panic.go:608 +0x5c
May 12 23:12:18 pi3plus influxd[30173]: runtime.largeAlloc(0x7e6c15dd, 0x60101, 0x76f91a20)
May 12 23:12:18 pi3plus influxd[30173]:         /usr/local/go/src/runtime/malloc.go:1021 +0x120
May 12 23:12:18 pi3plus influxd[30173]: runtime.mallocgc.func1()
May 12 23:12:18 pi3plus influxd[30173]:         /usr/local/go/src/runtime/malloc.go:914 +0x38
May 12 23:12:18 pi3plus influxd[30173]: runtime.systemstack(0x1c4e3c0)
May 12 23:12:18 pi3plus influxd[30173]:         /usr/local/go/src/runtime/asm_arm.s:354 +0x84
May 12 23:12:18 pi3plus influxd[30173]: runtime.mstart()
May 12 23:12:18 pi3plus influxd[30173]:         /usr/local/go/src/runtime/proc.go:1229
May 12 23:12:18 pi3plus influxd[30173]: goroutine 27 [running]:
May 12 23:12:18 pi3plus influxd[30173]: runtime.systemstack_switch()
May 12 23:12:18 pi3plus systemd[1]: influxdb.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
May 12 23:12:18 pi3plus systemd[1]: influxdb.service: Unit entered failed state.
May 12 23:12:18 pi3plus systemd[1]: influxdb.service: Failed with result 'exit-code'.

This happened because I’ve recently added statistics from my FritzBox with regards to my DSL line speed. The statistics have a high cadence, which means that many entries are created in influxdb in a short amount of time. Influxdb tries to create an index in RAM for these entries and is overwhelmed by the mass of data.

Therefore, I stopped the service with

sudo systemctl stop influxdb

and followed the suggestion from the upgrade instructions to use the influx_inspect tool.

I’ve executed influx_inspect as sudo and changed the permissions of my DB content folter later on with

chown -R influxdb:influxdb <folder>

This command may take a while to complete, depending on the size of your DB.

Once it is finished you can restart influxdb with

sudo systemctl start influxdb

Your server should now be stable again. The index is now disk based instead of being memory based, which could cause troubles on the limited resources of the Raspberry Pi.

Howto mass delete old Tweets on Twitter

There’s unfortunately no way to mass delete old Tweets you’ve posted on Twitter. There are some online services, who promise to delete your data for you, but since you’ll have to grant them access to your account I’ve got a bad feeling and wanted to do things on my own.

I’ve tried last year a windows only software called Twitter Archive Eraser. Last year it used to be a github project which you could compile locally and let it run on your account. It’s now free for a limited amount of tweets and also only works with tweets not older than two years. To remove these restrictions you’ve got to pay a small amount for a license.

You’ll need to download your complete message archive for the deletion process. Once you’ve got the data from Twitter you might as well start to write a little script which deletes the old messages for you using the Twitter post id.

Luckily, I found this blog post by Kris Shaffer. He explains how he deleted a large amount of his tweets using python so I’ve started to try this for myself. There was also a different blog which explained the process more beginner friendly. However, I’ve got problems with misformatted characters so I’ve decided to post my used code as gist to github:

To use this I’ve done the following things:

  • Requested and download my account data from Twitter
  • Create a Twitter developer account
  • Created a new app to get Api keys and Access tokens
  • Installed python3 on my mac with homebrew ‚brew install python3‘
  • Installed tweepy with pip3 ‚pip3 install tweepy‘
  • Created a virtual environment for this script
  • Copied the lines in blocks into the python3 interactive shell

Please be aware that above gist only deleted the tweets from 2017 to June 2018. Please refer for other scenarios to Kris blog post (e.g. delete only mentions in a given time frame).