Issues after Upgrade to Ubuntu 12.04 - Network Manager not starting and "Waiting for network configuration..." at boot
Learn the new Linux Network Commands
Copying Binary Files Via Telnet
On Terminal 1:
1. telnet host > somefile.txt (then log in blind or look at tail -f somefile.txt)
2. cat somefile | busybox uuencode -m -
3. Once you see that you are logged in and the data stream is going just wait for it to finish.
4. Close Telnet (Ctrl-C, Ctrl-D)
Then once telnet is closed and you are back on your system enter:
5. busybox uudecode -o somefile.bin somefile.txt
There is no need to first remove the login protocol and other random shell noise, they are no problem. uudecode automatically detects where the binary data starts and ends if you do one file at a time.
Saturating your Link with Lftp
Eduroam with Linux (incl. Android!)
Security: WPA2 Enterprise
Authentification Tunneled TLS (TTLS)
Anonymous Identity: none
CA-Certifikate: (none)
Inner Authentification: MSCHAP2
User: user@your-uni.com (as in the Windows settings guide of your home university)
password: your password, as in windows.
Syncronized Folders across Windows and Linux - (Cloud-)Free Encrypted Cross-Platform Synchronisation with Unison
What you need
The biggest problem is that you need a Linux server somewhere that is best constantly running and connected. I sync all my files with an online server. This is not a big deal because with SSH the connection is securely encrypted. Also due to compression and the fact that files are only transferred once, sync is pretty fast even with an ADSL connection. Of course this won't work for slow dial up connections in combination with large files. But as long as there are only small changes in certain files, even dial up might work for you. A Linux server is not really needed, but makes the setup much easier, which is why that's what I demonstrate here. And of course if you only want to sync two computers with each other, then there's no need for a constantly active server. It should work with Mac as a client or even server, but I don't own one so I can't help with that.
Downloads
You can get your unison downloads for Mac, Windows here and for Linux here (Ubuntu binaries i386 for 9.10 and 10.04 and more in this ppa. I recommend version 2.40.16, read below for why.
For Windows you also need to get Putty Agent (Pagent), Putty Puttygen and Plink get them at http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html. For Linux you also need openssh-server ssh-agent, ssh-keygen (in the openssh-client package) and keychain. You need to setup a working ssh connection with public key authentication for this to work without your interaction in the background. See here how to do this.
Bugs in the past
At the time I found it, over a year ago, it was still pretty buggy. It couldn't handle unicode, special characters or Umlauts properly. Especially between different system you'd end up with two differently named versions of a file unless you restricted yourself to normal a-z characters - very annoying.
But recently new version has come out, which fixes the problem. Now unison is not only a tool to synchronize flawlessly across different systems, it's also faster and prettier "GUI" than older versions and it still synchronized across encrypted ssh connections if you want it to. In my view this makes it the perfect tool for my needs. I've set it up to synchronize my laptop and my netbook with each other and my server system over ssh.
How it works
It's blazingly fast by only copying those parts of the files that have changed. And it doesn't transfer any files to check this. It runs both on the server and the client and only transfers the dates and hash coddes of the files via a secure compressed ssh connection. You can find out more about unison at LinuxJournal.
The easiest setup and the one I will show here is a star topology setup. This means there are several client systems synchronizing with one server. It works because unison knows which files are newer. The great side effect is that you automatically have a distributed backup on all your systems - the server and all clients of all your files.
The script runs automatically every X minutes in the background via cron on Linux (completely invisible) and is called by the Windows Task Manager on Windows (almost invisible).
Important to know
You need to use exactly the same version on all system you deploy it. There are easy to install packages for Debian and Ubuntu and precompiled versions for Windows. I haven't worked with OS X version yet, but they should work as well. I use a Linux server, Windows on the server side should be possible to set up but it's going to be much more difficult. Setting up the client on Windows is already non-trivial.
As you don't modify the same file on both systems at the same time before a server sync, then there won't be any problems. If you do, unison will not sync them. You need to call unison by itself and it will prompt you to chose how to deal with the situation.
The Linux version uses the symlink feature of Linux to let you configure which folders to sync on the go. In Windows this won't work, so all your folders you want to sync need to be inside one parent folder. I know no way around this, though the unison config file might have a solution somewhere...
Sync script:
#!/bin/bash
LOG="tee -a $HOME/logs/mysync2.log"
source $HOME/scripts/functions $HOME/scripts/variables
main ()
{
echo Starting at $(date)
# exit if running on battery
grep on-line /proc/acpi/ac_adapter/*/state >> /dev/null
if [ ! $? ]; # 1 if not "on-line"
then
echo Running on battery. Leaving now...
exit 0;
else
echo Running on AC Adapter.
fi
# use ssh-agent
source $HOME/.keychain/*-sh 2>&1
echo sock $SSH_AUTH_SOCK pid $SSH_AGENT_PID me $(whoami)
ssh-add -l || (echo "SSH Key not active\!"; ssh-add || exit 1)
cd $HOME
unison-2.40.16 sync -batch -maxbackups 2 ${@} 2>&1
echo Exiting at $(date)
}
main | $LOG
It checks and exits when the system is running on battery. To be able to work in the background, it uses ssh-agent. So you need to setup ssh key authentication.
Unison config in ~/.unison/sync.prf:
# Unison preferences file
root = /home/user/sync
root = ssh://user@syncserver:port/syncfolder
follow = Path *
ignore = Path {scripts/s3.sh}
This is a sample config file. It follows all symlinks in the $HOME/sync folder to syncfolder on syncserver via ssh. The ignore section shows you how to exclude certain files from synchronization.
Windows Setup
This should already work just fine in Linux. Now let's turn to the Windows client.
The Window setup, as you will see, it much more "fun". There are several steps you need: Putty Agent (Pagent), Putty Puttygen and Plink (see above for links) and then a few bash scripts.
You need to copy your private ssh key from your linux box and convert it to putty's format with puttygen. Then copy it into a safe folder. I have it in the same folder as the Unison.exe. Then setup an ssh connection to your server with Putty that uses the key and save it to profile name unisonssh. Copy the following bash files to your unison folder and adjust the path names accordingly.
Putty Agent.bat:
pageant.exe sshkey.ppkThis needs to be a script or otherwise it's not started in the right folder and won't find the key. Try it with the full path of the key file then. This must be linked into Autostart if your and will prompt you for your ssh key password if your key is password protected.
SSH Connect.bat
@plink -C -ssh -load unisonssh -i "C:\unison\sshkey.ppk" -l djtm -P 22 unison -server -autoChange the path to where your ssh key lies and the port to your server's port if necessary.
Unison Sync.bat
C:
cd "C:\unison\"
unison2.4.exe -sshcmd sshconnect.bat -backups -backupdir unisonbackups -backuplocation central -batch -confirmbigdeletes -contactquietly
Testing it
Now is a good chance to try everything works fine. Try creating a file one one system and see how after two syncs it magically appears on the other. Edit it there to see how the edits are transferred back to the first system.
Scheduling it
Once everything works you can schedule it to run regularly without you needing to do anything. In Linux, you should install keychain for this to work in scripts and then enter the following into your cron. (To edit your cron type crontab -e)
*/15 * * * * $HOME/scripts/unisonsyncThe */15 means to sync every 15 minutes. Don't worry, usually nothing will be done - I hardly ever notice anything happening. Of course the path should be where your unison sync script lies.
In Windows it's a tad more difficult again. The most difficult part of it all is to make it run in the background without annyoing you every 15 minutes or so. The following command looks a bit quirky, but that's the best I could do. Enter exactly this behind Execute: in the task scheduler.
cmd /C start /LOW /MIN "Unison Sync" "C:\unison\unison.bat"and "C:\unison" next to execute in. Check both execute only when logged in and, under the settings tab, not to start the task if you're running on battery.
And now - finally - enjoy great, free, in sync folders!
Another older, less detailed guide is available here and here. Thanks to the authors of Unison.
Fast, Cheap, Reliable VPN for Linux
You can sign up to the PureVPN shared IP services here. I have good experience (hulu with maximum quality streaming, iplayer work fine) with the cheapest version.
I recommend chosing the Monthly or Quarterly plan for tests ($14), but there is a three day test account ($3) available as well. It seems there is a bandwith limit now, and without you pay more.
Payment processing can take a few hours so be aware you can't start you fun and tests directly. Settings for network manager (gnome) can be found here. It works fine for me after installing network-manager-pptp-kde. Let me know in the comments if you need help. The most important part is to deselect everything in that one list except CHAP in the advanced settings and dselect MPPE. You can also try the nm-applet from Gnome, which you can of course also use in KDE. In my experience it often works better than the KDE version anyway unfortunately.
You may run into problems if your router doesn't support pptp forwarding. Then you either get a better router, or - in the long term probably the cheaper and better option for your speed - get a better router.
By signing up via a link from this blog you support the author.
Increase Speed and Save Traffic with Opera Turbo
Starting with Opera 10, there is the integrated "Opera Turbo". With the
help of that proxy you not only speed up the loading of websites by 4x and
more, but you also decrease the amount of data transfered, which makes
your volume last much longer.
After a whole day of 3G web surfing, Opera Turbo had saved me 2 GB of
traffic! (Only about 100 MB were left, that 20x less!) So the first thing
after setting up 3G should be setting up Opera with Opera Turbo!
Oh and you'll probably also want to put this list into your Opera urlfilter.ini. This stops Opera from loading ads, saving you even more traffic. But you can't use save as, you must use copy & paste to get rid of the html elements in the file.
Adjust Linux Network Receive Window for Fast Networks
sudo sysctl -w \
net.core.wmem_max=1075200 \
net.core.rmem_max=1075200 \
net.ipv4.tcp_rmem="4096 87380 8388608" \
net.ipv4.tcp_wmem="4096 87380 8388608" \
net.core.rmem_default=1075200 \
net.core.wmem_default=1075200
Install Daily Fresh Ubuntu Directly over the Web
Basically all you need to do is copy the kernel "linux" and the initrd "initrd.gz" to your stick. Then boot them from Grub and you can start your installation.
The great thing about it: During the installation you can chose whether to install Kubuntu, Xubuntu, Ubuntu, etc. And you will automatically get the newest available packages right away. There's no need to fetch and install updates after the installation.
And getting the newest packages right away might save you a lot of trouble because of bad packages in the installation cd. And the network installer itself also gets updated regularly.
Bye Bye Privacy
WPA Encryption Cracked
That means many people will need a new router and a new setup for it. And for a long time many networks will remain open again. If you want a long term solution it more and more looks like a VPN solution is the only way to go.
And if I'm not mistaken he who managed it is Erik Tews from the German University of Applied Sciences Darmstadt.
Okay, it turned out the hack is not as bad as thought. It will probably still take some time and combination with other approaches to really become useful. But I think this might help around the current restriction that you can only decypher clients that connected after you were "listening" with WPA.
A Tiny Backup Script with Rsync
#!/bin/sh
# (c) 2008 linux-tipps.blogspot.com
#
FROM=/home/user
# the base path
DIRS="Desktop Documents"
# you can also use absolute paths
RSYNC="rsync -av --progress -e ssh"
# add -z for compression
ssh-add -l || ssh-add
# this adds ssh private keys if not added already
# for cron scripts use keychain instead:
# source ~/.keychain/*-sh
# see here for more
cd $FROM
$RSYNC -z $DIRS djtm@server1:/some/path/Backup/ &
$RSYNC $DIRS djtm@server2:/some/path/Backup/ &
wait && echo Sync Done. || echo Sync Failed or Aborted.
Find Bots in your LAN with BotHunter
Their webpage has just gone offline. I'm sure that's because of all my readers visiting their web page right now. ;-)
Fixing Wake On LAN in Ubuntu 8.10
1. The network cards' drivers are unloaded during suspend.
This often deactivates your card's wake on lan feature and completely powers it off.
You have to add your network cards driver module to the MODULES_WHITELIST in /etc/default/acpi-support. Also add the network card's interface(e.g. eth0) to SKIP_INTERFACES.
2. The network is not (properly) restarted after resume.
a. NetworkManager overrides settings in /etc/network/interfaces.
The easy way is to simply remove networkmanager: sudo aptitude remove network-manager.
b. The network setup script is not restarted after resume.
You can add the script (networking) to STOP_SERVICES in /etc/default/acpi-support.
3. Wake on Lan is not enabled by default
You need to issue e.g. sudo ethtool -s eth0 wol g. You can do that in an init script and you will likely have to do it again after resume. (I've put it a script to do that into both /etc/acpi/suspend.d/54-wol.sh and /etc/acpi/resume.d/99-wol.sh )
Broadcom Wireless Drivers - wl vs. b43 - Performance Comparison
I've compared the performance of both drivers with a simple iperf test over a few meters:
Broadcom Proprietary Driver:
TCP
------------------------------------------------------------
TCP window size: 85.3 KByte (default)
------------------------------------------------------------
[ 4] local 192.168.178.24 port 5001 connected with 192.168.178.34 port 46362
[ 4] 0.0-10.2 sec 24.6 MBytes 20.2 Mbits/sec
[ 5] local 192.168.178.24 port 5001 connected with 192.168.178.34 port 46363
[ 5] 0.0-10.2 sec 24.6 MBytes 20.2 Mbits/sec
[ 4] local 192.168.178.24 port 5001 connected with 192.168.178.34 port 46365
[ 4] 0.0-10.2 sec 24.0 MBytes 19.8 Mbits/secUDP
------------------------------------------------------------
Receiving 1470 byte datagrams
UDP buffer size: 375 KByte (default)
------------------------------------------------------------
[ 3] local 192.168.178.24 port 5001 connected with 192.168.178.34 port 35141[ 3] 0.0-10.0 sec 24.2 MBytes 20.2 Mbits/sec 1.364 ms 0/17239 (0%)[ 3] 0.0-10.0 sec 1 datagrams received out-of-order[ 4] local 192.168.178.24 port 5001 connected with 192.168.178.34 port 38437[ 4] 0.0-10.0 sec 27.2 MBytes 22.8 Mbits/sec 1.837 ms 0/19418 (0%)[ 4] 0.0-10.0 sec 1 datagrams received out-of-order[ 3] local 192.168.178.24 port 5001 connected with 192.168.178.34 port 45356[ 3] 0.0-10.0 sec 28.8 MBytes 24.1 Mbits/sec 1.319 ms 0/20555 (0%)[ 3] 0.0-10.0 sec 1 datagrams received out-of-order
B43 Open Source DriverTCP
------------------------------------------------------------
TCP window size: 85.3 KByte (default)
------------------------------------------------------------
[ 4] local 192.168.178.24 port 5001 connected with 192.168.178.34 port 42345
[ 4] 0.0-10.1 sec 19.6 MBytes 16.4 Mbits/sec
[ 5] local 192.168.178.24 port 5001 connected with 192.168.178.34 port 42346
[ 5] 0.0-10.2 sec 23.1 MBytes 19.0 Mbits/sec
[ 4] local 192.168.178.24 port 5001 connected with 192.168.178.34 port 42347
[ 4] 0.0-10.2 sec 21.7 MBytes 17.8 Mbits/secUDP
------------------------------------------------------------
Receiving 1470 byte datagrams
UDP buffer size: 375 KByte (default)
------------------------------------------------------------[ 3] local 192.168.178.24 port 5001 connected with 192.168.178.34 port 57459
[ 3] 0.0-10.1 sec 14.0 MBytes 11.7 Mbits/sec 0.438 ms 152/10171 (1.5%)
[ 3] 0.0-10.1 sec 1 datagrams received out-of-order
[ 4] local 192.168.178.24 port 5001 connected with 192.168.178.34 port 55062
[ 4] 0.0-10.0 sec 15.7 MBytes 13.1 Mbits/sec 1.372 ms 205/11373 (1.8%)
[ 4] 0.0-10.0 sec 1 datagrams received out-of-order
[ 3] local 192.168.178.24 port 5001 connected with 192.168.178.34 port 49861
[ 3] 0.0-10.1 sec 23.4 MBytes 19.5 Mbits/sec 3.724 ms 292/17017 (1.7%)
[ 3] 0.0-10.1 sec 1 datagrams received out-of-order
[ 4] local 192.168.178.24 port 5001 connected with 192.168.178.34 port 42069
[ 4] 0.0-10.0 sec 21.5 MBytes 18.0 Mbits/sec 1.370 ms 239/15544 (1.5%)
[ 4] 0.0-10.0 sec 1 datagrams received out-of-order
As you can see, the proprietary driver is slightly faster in TCP mode. And somehow they're both really slow in UDP mode. That's something I'd like to find out more about.
Update: That was my fault. For some reason iperf restricts UDP tests to 1 Mbit/s. Now the results are more appropriate. The result is that the proprietary driver has a more stable connection speed.
But I think that's really not worth installing a proprietary driver. Broadcom really would better integrate the changes into the open source driver. Their driver is still relying on the deprecated wireless system anyway.
Update2: Further tests showed that, at least after a while of usage, the proprietary driver creates high network latencies. Ping times to the router went up from 2 ms to an average of 160ms (peaks of 500ms !).
Broadcom Proprietary Drivers with Linux 2.6.27
Then unpack, go to ubuntu-intrepid-lrm/ubuntu-restricted/broadcom
and make -C /lib/modules/`uname -r`/build M=`pwd`
They only work with TKIP for me, at least not with WPA2. (The open source b43 drivers does.)