Increasing Network Speed with the example of Raspberry Pi4 and Raspbian

I could increase the maximum internet download speed of my Raspberry Pi4 running with the most current Raspbian 64 bit kernel from 60-70 MB/s to a bit over 90 MB/s with the following adjustments. Please note many servers limit downloads e.g. to 400 Mbit/s, even speed test servers.

1. A bash script - you could post this into /etc/rc.local:

# network speed
iface=eth0

# speed up internet - https://datatag.web.cern.ch/datatag/howto/tcp.html
/sbin/ifconfig $iface txqueuelen 8000
echo 8000 > /proc/sys/net/core/netdev_max_backlog
ip link set $iface qlen 20000

# google bbr - https://www.tecmint.com/increase-linux-server-internet-speed-with-tcp-bbr/
sysctl -w net.core.default_qdisc=fq
sysctl -w net.ipv4.tcp_congestion_control=bbr

# https://www.cyberciti.biz/faq/rhel-centos-fedora-debian-configure-rx-polling-mode/
ethtool -C $iface adaptive-rx on

# https://cromwell-intl.com/open-source/performance-tuning/ethernet.html
ethtool -K $iface tx-checksum-ipv4 on
ethtool -K $iface tx-checksum-ipv6 on

# not supported https://cromwell-intl.com/open-source/performance-tuning/ethernet.html
# ethtool -A $iface rx on
# ethtool -A $iface tx on
# ethtool -G $iface rx 4096 tx 4096

Kernel tweaks, you could put this in /etc/sysctl.d/99-netspeed.conf

# https://serverfault.com/a/758350
kernel.sem = 350 358400 64 1024
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 4194304
net.ipv4.tcp_window_scaling = 1
net.ipv4.tcp_adv_win_scale = 2
net.ipv4.tcp_moderate_rcvbuf = 1
net.ipv4.tcp_rmem = 4096 262144 4194304
net.ipv4.tcp_wmem = 4096 262144 4194304
net.ipv4.tcp_keepalive_time = 900
net.ipv4.tcp_keepalive_intvl = 900
net.ipv4.tcp_keepalive_probes = 9

3. Installing irqbalance:

sudo apt install irqbalance

4. Reboot

Enjoy!

Enable fstrim / TRIM / discard support especially for USB SATA disks in Linux

If fstrim tells you "discard operation is not supported" on a usb disk then you may want to try this tip. I'm assuming you're using /dev/sdb below, please adjust.

Check if this commands

sg_readcap -l /dev/sdb | grep unmap

shows "Unmap command supported (LBPU): 1". It must be  a 1 at some point or you're out of luck.

If that works. Try unmounting the drive. Then try this command as root:

umount /media/disk

echo unmap > /sys/block/sdb/device/scsi_disk/*/provisioning_mode

Now try again to see if it works:

mount /dev/sdb2 /media/disk
fstrim -v /media/diskidea

If that works for you, find out the device ID for the usb device with:

lsusb

and then post the IDs into a udev rule (exampe ID is the StarTech USB 3.0 to 2,5" SATA cable, which I've bought for my Raspberry Pi 4):

echo 'ACTION=="add|change", ATTRS{idVendor}=="174c", ATTRS{idProduct}=="55aa", SUBSYSTEM=="scsi_disk", ATTR{provisioning_mode}="unmap"' > /etc/udev/rules.d/99-trim-ssd-discard.rules

Hope this works for you! Thanks to the original idea!

Fritz Box VPN mit Cisco und IKE-Fehler

Hier zeige ich wie man eine Fritz Box mit einen Cisco-VPN verbinden kann. Damit könnte man grds. wohl auch VoIP über VPN bewerkstelligen.

Ich habe folgende Grundkonfiguration für Cisco-VPN genutzt, einfach anpassen und in die Fritz Box importieren:

vpncfg {
        connections {
                enabled = yes;
                editable = no; // darf nicht an sein, zerschießt sonst die Einstellungen
                conn_type = conntype_lan;
                name = "dus.net";
                always_renew = yes;
                reject_not_encrypted = no;
                dont_filter_netbios = yes; // muss immer aktiv sein
                localip = 0.0.0.0;
                local_virtualip = 0.0.0.0;
                remoteip = ...;
                remote_virtualip = 0.0.0.0;
                localid {
                        key_id = "gruppenname";
                }
                mode = phase1_mode_aggressive;
                phase1ss = "alt/all/all";
                keytype = connkeytype_pre_shared;
                key = "gruppenschlüssel";
                cert_do_server_auth = no;
                use_nat_t = yes;
                use_xauth = yes;
                xauth {
                        valid = yes;
                        username = "";
                        passwd = "";
                }
                use_cfgmode = yes; // automatische Einrichtung des remote network
                phase2localid {
                        ipnet {
                                ipaddr = 192.168.178.0; // Netz der Fritz Box
                                mask = 255.255.255.0;
                        }
                }
                phase2ss = "esp-aes256-3des-sha/ah-no/comp-lzs-no/pfs";
                accesslist =
                             "permit ip any 1.2.3.0 255.255.255.0", // remote network
                             "permit ip any host 3.4.5.6"; // einzelner host
        }
        ike_forward_rules = "udp 0.0.0.0:500 0.0.0.0:500",
                            "udp 0.0.0.0:4500 0.0.0.0:4500";
}

​​AVM hat eine Liste der Bedeutung der diversen IKE-Fehler, aber sie schreiben leider nicht, was man zur Behebung anpassen muss. Ich versuche das stellenweise zu ergänzen. Grundlageninformationen finden sich auch hier:

• Einrichtung mit Juniper: https://blog.webernetz.net/2013/12/02/ipsec-site-to-site-vpn-juniper-screenos-avm-fritzbox/
  
• Liste der IKE-Fehler bei AVM: https://service.avm.de/help/de/FRITZ-Box-7362-SL-avm/016/hilfe_syslog_122
  
• Sicherheitseinstellungen: https://blog.webernetz.net/2015/03/11/fritzos-ab-06-23-ipsec-p2-proposals-erweitert
• Probleme nach dem Update zu Version 6.2 (wohl behoben): https://blog.webernetz.net/2014/11/25/fritzos-ab-06-20-anderungen-bei-vpns/
• Weitere Beiträge im Webernetz: https://blog.webernetz.net/tag/fritzbox/
  

Typische Fehler

IKE-Error 0x2020 "hash mismatch in received packet"

Hier gibt es einen Fehler bei den Gewählten Algorithmen (vgl. "Tests") wohl in Phase 1 oder das Passwort stimmt nicht. Bei mir hat diese Anpassung geholfen:

phase1ss = "alt/all/all";

 

​​Probieren kann man diese:

 

dh5/aes/sha
dh14/aes/sha
dh15/aes/sha
def/all/all
alt/all/all
all/all/all
LT8h/all/all/all

Angeblich kann auch teilweise die MTU schuld sein an diesem Fehler. Sie lässt sich bei der Fritz Box allerdings nicht anpassen. Es kann wohl auch an einem falschen key oder der falschen key_id liegen.

IKE-Error 0x2026 "no proposal chosen"

Hier handelt es sich wohl um Fehler in Phase 2. 

Es stehen u.a. folgende Alternativen zur Verfügung:

esp-3des-sha/ah-no/comp-no/pfs
esp-3des-sha/ah-no/comp-no/no-pfs
esp-aes256-3des-sha/ah-no/comp-lzs-no/pfs
esp-aes-sha/ah-all/comp-lzjh-no/pfs
esp-all-all/ah-all/comp-all/pfs
esp-all-all/ah-all/comp-all/no-pfs
esp-all-all/ah-none/comp-all/pfs
esp-all-all/ah-none/comp-all/no-pfs
LT8h/esp-all-all/ah-none/comp-all/pfs
LT8h/esp-all-all/ah-none/comp-all/no-pfs

Noch ein paar Notizen

• Debugging geht über System -> Ereignisse.
• Die accesslist bestimmt die Routen, die über das VPN gehen: any any heißt von allen Geräten zu allen. Das bedeutet, es wird die default route. Das will man oft nicht.
  
• Wenn die Fritz Box in Ereignisse gar keine Fehler anzeigt, liegt meist ein grober Konfigurationsfehler vor. Die Fritz Box versucht dann gar nicht erst, eine Verbindung herzustellen.
  
• Die Einrichtung des Cisco-Routers wird auch hier beschrieben: https://blog.webernetz.net/2014/06/25/ipsec-site-to-site-vpn-cisco-router-avm-fritzbox/
  
• Die Parameter der Konfigurationsdatei werden hier teilweise erläutert: http://www.wehavemorefun.de/fritzbox/Talk:Versteckte_Features
  
• Das Routing wird hier gut von AVM beschrieben: https://en.avm.de/service/fritzbox/fritzbox-6840-lte/knowledge-base/publication/show/230_Accessing-multiple-IP-networks-behind-a-FRITZ-Box-over-VPN-connection-between-two-FRITZ-Boxes/. Hier finden sich hilfreiche beispiele (nach "permit" suchen): http://www.wehavemorefun.de/fritzbox/Ar7.cfg
  
• Man darf Netbios nie filtern, sonst verbindet sich die Fritz Box gar nicht erst.
  
• Es kann auch helfen, die Einstellungen der Fritz Box zu exportieren und sich dort Beispielkonfigurationen anzusehen.
• Es gibt auch einen passenden Artikel zu Fritz Box VPNs mit Firewallsin einer aktuellen c't: "In Phase 1 und Phase 2 werden außerdem die Krypto-Protokolle (Proposals) und die Dauer der Schlüsselgültigkeit ausgehandelt (Lifetimes). Dieser Einstellungsbereich gilt als Mutter allen IPsec-Übels – weil es sehr viele Optionen gibt."

Samsung Scanner in Your Network Despite Firewall With Manual Setup

​To manually detect a Samsung network scanner with sane, you can try this method:

echo 'tcp scx' | sudo tee -a /etc/sane.d/xerox_mfp.conf

Replace "scx" with ​the IP address or resolvable hostname of your scanner. Try to ping the host, e.g. "ping scx" before you enter it here.

This assumes using the drivers from www.bchemnet.com/suldr. For some services, you may have to restart the service or your computer. It fixed using the scanner with an active firewall for me.

Diverse Probleme mit der Fritz Box bei fehlerhaften VPN-Einstellungen

Wenn man die VPN-Funktion der Fritz Box nutzt, sollte man regelmäßig prüfen, ob alle ausgehenden Verbindungen funktionieren.

Denn nicht funktionierende ausgehende VPN-Verbindungen (z.B. Kopplung zwischen zwei Fritz Boxen) können zu diversen Problemen führen, u.a. schlechte Sprachqualität beim VoIP, Paketverluste und langsameres Internet. Wenn man eine aktive VPN-Verbindung hat, die die Default Route setzt (z. B. automatisch via use_cfgmode = no;), kann sogar das Internet insgesamt unterbrochen werden.

Die Lösung ist einfach: Den Haken links neben der VPN-Verbindung abwählen und die Einstellungen übernehmen.

[Fix] Getting the MSI PE60 6QE (Skylake) to run Linux

After lots of experimentation, it turned out that the best boot parameter so far (Ubuntu 16.10 alpha July 2016) is:

pci=noacpi

Otherwise (e.g. with acpi=noirq), there is a crash trying to load the nvidia card and an issue with ACPI trying to load _DSM. If you want to disable the nivida card completely, for 6W or more of power savings, use nouveau.modeset=0. This might need to be in addition to pci=noacpi.

This might work for lots of other notebooks, especially with Nvidia graphics and a Skylake CPU, e.g. the MSI GE62 6QF series.

Prioritizing System Services with Systemd - Run a Process Permanently in the Background

If you simply want to adjust some priorities, here is how to do it in three steps.

1. sudo systemctl edit [service]
2. sudo systemctl daemon-reload
3. sudo systemctl restart [service]

Removing all images with rkt

Note this will remove all images, not just stale ones. Set rkt to point to your rkt binary.

#!/bin/sh
rkt="./rkt/rkt"
images="$(sudo $rkt image list | awk 'NR>1 {print $1}')"
set -x
sudo $rkt gc --grace-period=1s
sleep 1
sudo $rkt gc --grace-period=1s
sudo $rkt image list
sudo $rkt image rm $images

Forward Wake on LAN packages bewteen networks with socat

If you have two networks and you want to wake a machine in another from network it can be tricky because you can't always set the destination IP address or network. An easy fix is to use socat to forward the UDP wakeonlan packages to the other network or IP:

sudo socat -v UDP4-RECVFROM:9,fork UDP4-SENDTO:192.168.1.255:9

You can leave out the -v for less verbosity. The target network is 192.168.1.255, you can replace this with a specific IP, which helps with routers not forwarding broadcasts. Usually UDP port 9 is used, but you may change this to port 7 for your setup. You will need to be root because port 9 is generally privileged.

Fscking Precautions: Snapshots and the undo file

If you have a badly corrupted filesystem, e.g. because you had back blocks on the hard drive, you have want to take some precautions to make sure fsck doesn't destroy your files. This goes especially for large raids.

Something you can always do is test the fsck operation on a dmsetup snapshot. For this to work you must boot from a different partition and it must have a sparse file enabled filesystem.

# The path to your snapshot storage file.
INPUT=/dev/sdb4
COW=

​/root/sdb4-​

​snapshot

#

​20​

Terabytes should be enough for your partition, otherwise increase this number
truncate -s

​20​

000G $COW 
#setup a loop for dmsetup snapshots to the COW file.
loop="$(losetup -f)"
losetup $loop $COW
#setup the snapshot device
echo 0 `blockdev --getsz $INPUT` snapshot $INPUT $loop p 8 | dmsetup create top
# let you know where the snapshot device is.
echo loop: $loop top: /dev/mapper/top
dmsetup status

After this you should be able to fsck /dev/mapper/top. You can see how much space the COW file actually occupies with du -h $COW. You may also want to get the newest fsck version (e.g. with a newer fsck-static package). If you end up with many multiply-claimed blocks, this e2fsck version may help: http://git.hpdd.intel.com/tools/e2fsprogs.git/ (checkout a -wc branch).

Good luck, you might need it!

Linux Raid: ignoring /dev/sdX as it reports /dev/sdY as failed

The fix for this problem may be extremely easy. What happened is that some disks of the raid failed. They were ejected. This happens. But the raid won't be assembled anymore if the failed disks are first on the mdadm assemble command line. Because for some reason, mdadm does not check what most disks say, but what the first disks say. So if you have a raid with 10 disks and the first two on the command line are failed, it will reject the remaining 8, because the are not compatible. All you need to do now is to list those two failed disks at the end with --force to activate the raid again:

Instead of 

mdadm /dev/md1 --assemble /dev/sdX,Y /dev/sd[a-f]

try

mdadm /dev/md1 --assemble /dev/sd[a-f] /dev/sdX,Y

Note that there's probably still a good reason for those disks to have been marked as failed...

Windows 7 Detects only Some of the CPUs added in KVM [Fix]

If Windows XP, Windows 7, maybe even Windows 8 or later don't detect all your CPUs in KVM, you may need to change the settings. Windows often doesn't like if you have too many cpu sockets. Try a configuration with 1 or 2 sockets and several cores.

How to Dynamically Switch Between Uniprocessor and SMP during Windows XP boot

If you're running Windows XP inside a Virtual Machine such as VirtualBox, Vmware or KVM, you may want to sometimes boot with only one active CPU, other times with several CPUs. If you don't set up windows correctly, it will neither boot normally, nor in safe mode (where it will stop with a blank screen and a blinking cursor).

But you can edit your boot.ini to look like this in order to dynamically switch between one and several processors in Windows XP:

[boot loader]
timeout=5
default=multi(0)disk(0)rdisk(0)partition(1)\WINDOWS
[operating systems]
multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Microsoft Windows XP Pro" /noexecute=optin /fastdetect
multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Microsoft Windows XP Pro one processor" /noexecute=optin /fastdetect /kernel=ntoskrnl.exe /HAL=Halaacpi.dll
multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Microsoft Windows XP Pro SMP" /noexecute=optin /fastdetect /kernel=ntkrnlmp.exe /hal=halmacpi.dll

The Ultimate Setup Guide for ownCloud on Small Systems such as the Raspberry Pi

It took me about a year to collect this information. There are many guides, but all I found are incomplete. Here's the one guide to rule them all - hopefully. The guide works on Ubuntu and Debian without changes. It's optimized for resources, speed, security and ease of use. While this runs well on my old phone (ARMv7; 512 MB; ~1000 BogoMIPS), it should run even better on a Raspberry Pi.

Accessing Public ownCloud shares via WebDAV

Since ownCloud supports server to server sharing you can access public shares via webdav clients. You simply use the access token (t=...) as user name. If there is a password, use that as password, otherwise leave it blank. As URL you use the owncloud address with /public.php/webdav at the end:

https://myserver.com/owncloud/public.php?service=files&t=12345

-> webdavs://12345@myserver.com/owncloud/public.php/webdav

Speeding up your ownCloud on small systems such as the Raspberry Pi with Sqlite

I had seriously considered to use a raspberry pi system for setting up my ownCloud server. But the old pi had pretty much the same capabilities as my old mobile phone with cyanogen. And the phone has Wifi and some flash space already included. So I opted for the phone. As with most pi installations, the performance was not great. But I found some easy tweaks I haven't seen anywhere else to significantly improve the performance. I'm assuming you're already using the usual tweaks such as opcache(this will usually help more than the following tweaks!) and using cron.php. Please backup your owncloud.db before you start! You will need the sqlite3 tool (sudo apt-get install sqlite3).

Two ways to download your pictures from Picasaweb with Linux

There is a nerdy and a simple way. The simple way is to use the google data export ("takeout") pages. You select your photos, create an archive and download it. You may have to upgrade from Picasaweb to Google+ in order for this to work properly.

The nerdy way is via googlecl. Unfortunately this option does not currently work properly due to googlecl still using oauth version 1. But once that's fixed - or if you're still signed into googecl - you can use these two commands:

google picasa list-albums > albums.txt
parallel -a "albums.txt" -j 3 --eta google picasa get "{,}" .

Beware of weird folder names including / or other special symbols. Now you can e.g. move your pictures to your ownCloud server. No picasa needed.

How to change the volume with an active lock screen in android lollipop

This works at least from my Samsung galaxy S4: you press the power button for a few seconds until you see the pop-up to turn off the device, or to go into offline mode. Now you can use the volume buttons to change the ring tone volume.

Fixing Netflix Error Code: M7361-1254 on a Linux System

If you get this error code there is an issue with the sound output. If you use pulse, kill the pulseaudio process. Then close the browser, open it again.

Otherwise remove the kernel driver module for your sound card and insert it again. Then close and open the browser.

How to Fix a common VirtualBox Segfault

If you have a segmentation fault during the start of a VirtualBox machine, you are likely using an old, incompatible version of the VirtualBox Extensions. Simply update or uninstall them and things should work again.

An example from the dmesg kernel log:

EMT-1 [7265]: segfault at 618 ip 00007f0eaacbef31 sp 00007f0ed2afbc70 error 4 in VBoxDD.so [7f0eaac00000+26f000].