Category Archives: Networking

Disabling “secured” IPv6 addresses is macOS 10.12 Sierra

On older macOS versions, every network interface would have one IPv6 address autogenerated from its MAC address, easily identified by the characteristic “ff:fe” bytes in the middle of the host part:
$ ifconfig en0
[...]
ether 10:dd:b1:9f:6b:ba
inet6 fe80::12dd:b1ff:fe9f:6bba%en0 prefixlen 64 scopeid 0x4
inet6 2001:7c0:2012:4a:12dd:b1ff:fe9f:6bba prefixlen 64 autoconf
[...]

Since macOS 10.12 however, these were replaced with randomly-generated “secured” addresses:
$ ifconfig en0
[...]
ether 10:dd:b1:9b:d0:67
inet6 fe80::46:3b36:146:9857%en0 prefixlen 64 secured scopeid 0x4
inet6 2001:7c0:2012:4a:4e6:f1d1:dd90:c6b4 prefixlen 64 autoconf secured
[...]

Very little is known about these, besides a single mailing list post that discovered them. If you are running a server, you’ll want your IPv6 address to be deterministic so you can register it in DNS. Therefore, we need to revert to pre-10.12 behavior:

$ echo net.inet6.send.opmode=0 >> /etc/sysctl.conf
$ reboot

If you look at the source code of the XNU kernel (Search for the IN6_IFF_SECURED flag) and the IPConfiguration service in macOS 10.11 (the 10.12 source code hasn’t been released yet), you can see that the new behavior was already there, just not enabled by default like it is now. Also, we now know that the change wasn’t made to reflect RFC 7217 (Semantically Opaque Interface Identifiers) behavior, but rather implements RFC 3972 (Cryptographically Generated Addresses).

Using a BIND DNS server in an Active Directory Environment

Years ago, I posted a script that allowed ISC DHCPd to update a Microsoft DNS server with dynamic records for DHCP clients. I haven’t used that method in a long time and there is a much simpler method: use ISC DHCPd together with the BIND DNS server like everybody else does, and only delegate the _mscds and _sites zones from the BIND server to the Microsoft DNS servers:

_msdcs.example.com. 86400 IN NS dc01.example.com.
_msdcs.example.com. 86400 IN NS dc02.example.com.
_sites.example.com. 86400 IN NS dc01.example.com.
_sites.example.com. 86400 IN NS dc02.example.com.

Then on all your machines, use the BIND server as DNS server (typically set via DHCP option 23). For Windows Domain matters, only records below _msdcs and _sites are ever looked up.

I believe you should even be able to point your domain controllers to the BIND DNS server — they should be able to follow the NS record so that whenever they try to update their own records, they do so on the Microsoft DNS server. As it turns out, the RFC 2136 DNS UPDATE method is used when domain controllers try to register their own records, so you’ll see error messages in your logs if you point your domain controllers to the BIND DNS server (on a Microsoft DC, these would refer to NETLOGON and dynamic DNS registrations, while on a Samba DC they would be about samba_dnsupdate). If you are running Samba 4.5 or higher, you should ensure that samba_dnsupdate is called with the –use-samba-tool flag, which can probably be done by setting the option below in your /etc/samba/smb.conf. If you are running an older Samba version or any Windows Server version, you need to resort to using your domain controllers’ IP addresses as DNS servers on on all domain controllers (Samba: put them into /etc/resolv.conf, Windows: set them in the network interface properties).

dns update command = /usr/sbin/samba_dnsupdate --use-samba-tool

For compatibility with Unix clients (including Mac OS X), you’ll want to add a couple of CNAME records for the SRV records:

_ldap._tcp.example.com. 86400 IN CNAME _ldap._tcp.dc._msdcs.example.com.
_gc._tcp.example.com. 86400 IN CNAME _ldap._tcp.gc._msdcs.example.com.
_kerberos._tcp.example.com. 86400 IN CNAME _kerberos._tcp.dc._msdcs.example.com.
_kerberos._udp.example.com. 86400 IN CNAME _kerberos._tcp.dc._msdcs.example.com.
_kpasswd._tcp.example.com. 3600 IN SRV 0 100 464 dc01.example.com
_kpasswd._tcp.example.com. 3600 IN SRV 0 100 464 dc02.example.com
_kpasswd._udp.example.com. 3600 IN SRV 0 100 464 dc01.example.com
_kpasswd._udp.example.com. 3600 IN SRV 0 100 464 dc02.example.com

The _kpasswd records unfortunately can’t be CNAMEs because they don’t exist in the _msdcs branch, so you manually need to keep them up-to-date when you add and remove domain controllers.

Snom VoIP phones: OpenVPN and multicast audio

Snom voice-over-IP phones have a built-in OpenVPN client, and they can also have audio transmitted to them via multicast. However, in my case a phone logged into OpenVPN did not play back audio received from the local ethernet network

I had multicast reception configured correctly on the phone:

multicast_listen=on
mc_address=239.255.255.245:5555

(Remember that you need to push the Apply button at the bottom of the page and then the Save button at the top of the page for the setting to actually take effect.)

To diagnose the problem, you can use multicast pings. Running

ping -I eth0 -t 2 239.255.255.245

on the local network showed no reactions from the phone. However, running

ping -I tun2 -t 2 239.255.255.245

on the VPN server showed responses from the phone. So clearly the phone wanted to do multicast exclusively via VPN. The cause of this was the following line in my OpenVPN config on the server:

push "redirect-gateway def1"

The obvious solution was to replace this line with routes that excluded the multicast IP addresses:

push "route 0.0.0.0 128.0.0.0"
push "route 128.0.0.0 192.0.0.0"
push "route 192.0.0.0 224.0.0.0"

While this worked, the phone wouldn’t complete booting up the next time I power-cycled it. So I switched back to the old OpenVPN config and instead turned on multicast routing on the VPN server:

apt-get install smcroute
echo 'mgroup from eth0 group 239.255.255.245' > /etc/smcroute.conf
echo 'mroute from eth0 group 239.255.255.245 to tun0 tun1 tun2' > /etc/smcroute.conf
service smcroute restart

Multicast audio now works and it even goes across the VPN.

Below you can find some commands that use FFMPEG to send the multicast audio:

ffmpeg -re -i song.mp3 -filter_complex 'aresample=16000,asetnsamples=n=160' -acodec g722 -ac 1 -vn -f rtp udp://239.255.255.245:5555
ffmpeg -re -i song.mp3 -filter_complex 'aresample=8000,asetnsamples=n=160' -acodec pcm_mulaw -ac 1 -vn -f rtp udp://239.255.255.245:5555
ffmpeg -re -i song.mp3 -filter_complex 'aresample=8000,asetnsamples=n=160' -acodec pcm_alaw -ac 1 -vn -f rtp udp://239.255.255.245:5555

Interestingly, you don’t even need to set the rtp_codec_type setting — the phone automatically determines the codec from the stream. I wasn’t able to get the Opus codec working though, the phone just makes crackling noises when I tried.

PHP 5: ldap_search never returns when searching Active Directory

I recently moved a PHP web application from a server running PHP 5.3 on Mac OS X 10.6 to a newer one with PHP 5.4 on Mac OS X 10.9. This caused the following code sample, run against an Active Directory server, to hang at the ldap_search() call:

$conn = ldap_connect('ldaps://' . $LDAPSERVER);
ldap_set_option($conn, LDAP_OPT_PROTOCOL_VERSION, 3);
$bind = @ldap_bind($conn, $LDAPUSER, $LDAPPW);
$result = ldap_search($conn, $LDAPSEARCHBASE, '(&(samaccountname=' . $searchuser . '))');
$info = ldap_get_entries($conn, $result);
ldap_close($conn);

Wiresharking the connection between web server and LDAP server (after replacing ldaps:// with ldap://) showed:

bindRequest(1) "$LDAPUSER" simplebindResponse(1) success searchRequest82) "$LDAPSEARCHBASE" wholeSubtree
searchResEntry(2) "CN=$searchuser,...,$LDAPSEARCHBASE" | searchResRef(2) | searchResDone(2) success [1 result]
bindRequest(4) "" simple
bindResponse(4) success
searchRequest(3) "DC=DomainDnsZones,$LDAPSEARCHBASE" wholeSubtree
searchResDone(3) operationsError (000004DC: LdapErr: DSID-0C0906E8, comment: In order to perform this operation a successful bind must be complete on the connection., data0,

So it’s binding, receiving a success response, searching and then receiving a response and a referrer to DC=DomainDnsZones,$LDAPSEARCHBASE. Next, it opens a new TCP connection and follows the referrer, but does an anonymous bind.

The solution is simple: just add

ldap_set_option($conn, LDAP_OPT_REFERRALS, FALSE);

after line 2. If for some reason you actually need to follow the referrer, have a look at ldap_set_rebind_proc, which lets you specify a callback which then does the authentication upon rebind.

Update August 2015: Same goes when using Net_LDAP3, which is used e.g. by Roundcube’s LDAP integration. Here you need to add the following:

$config['ldap_public']['public'] = array(
[...]
 'referrals' => false,
);

VMware ESXi 5.5.0 panics when using Intel AMT VNC

For compatibility with a new guest OS, I upgraded my ESXi  to 5.5 today. During reboot, it crashes after a few seconds (it briefly flashes a message about starting up PCI passthrough on the yellow ESXi boot screen). The purple screen of death (PSOD) I get looks like this:

VMware ESXi 5.5.0 [Releasebuild-1474528 x86_64]
#PF Exception 14 in world 32797:helper1-2 IP 0x4180046f7319 addr 0x410818781760
PTEs:0x10011e023;0x1080d7063;0x0;
cr0=0x8001003d cr2=0x410818781760 cr3=0xb6cd0000 cr4=0x216c
frame=0x41238075dd60 ip=0x4180046f7319 err=0 rflags=0x10206
rax=0x410818781760 rbx=0x41238075deb0 rcx=0x0
rdx=0x0 rbp=0x41238075de50 rsi=0x41238075deb0
rdi=0x1878176 r8=0x0 r9=0x2
r10=0x417fc47b9528 r11=0x41238075df10 r12=0x1878176
r13=0x1878176000 r14=0x41089f07a400 r15=0x6
*PCPU2:32797/heler1-2
PCPU  0: SSSHS
Code start: 0x418004600000 VMK uptime: 0:00:00:05.201
0x41238075de50:[0x4180046f7319]BackMap_Lookup@vmkernel#nover+0x35 stack: 0xffffffff00000000
0x41238075df00:[0x418004669483]IOMMUDoReportFault@vmkernel#nover+0x133 stack: 0x60000010
0x41238075df30:[0x418004669667]IOMMUProcessFaults@vmkernel#nover+0x1f stack:0x0
0x41238075dfd0:[0x418004660f8a]helpFunc@vmkernel#nover+0x6b6 stack: 0x0
0x41238075dff0:[0x418004853372]CpuSched_StartWorld@vmkernel#nover+0xf1 stack:0x0
base fs=0x0 gs=0x418040800000 Kgs=0x0

When rebooting the machine now, it reverts to my previous version, ESXi 5.1-914609.

A bit of playing around revealed: This only happens if I am connected to the Intel AMT VNC server. If I connect after ESXi has booted up, it crashes a fraction of a second after I connect to VNC. Go figure! Apparently it’s not such a good idea to have a VNC server inside the GPU, Intel…

Before I figured this out, I booted up the old ESXi 5.1.0-914609 and even upgraded it to ESXi 5.1.0-1483097.  Looking at dmesg revealed loads of weird errors while connected to the VNC server:

2014-02-13T11:23:15.145Z cpu0:3980)WARNING: IOMMUIntel: 2351: IOMMU Unit #0: R/W=R, Device 00:02.0 Faulting addr = 0x3f9bd6a000 Fault Reason = 0x0c -> Reserved fields set in PTE actively set for Read or Write.
2014-02-13T11:23:15.145Z cpu0:3980)WARNING: IOMMUIntel: 2371: IOMMU context entry dump for 00:02.0 Ctx-Hi = 0x101 Ctx-Lo = 0x10d681003

lspci | grep ’00:02.0 ‘ shows that this is the integrated Intel GPU (which I’m obviously not doing PCI Passthrough on).

So

  • ESXi 5.5 panics when using Intel AMT VNC
  • ESXi 5.1 handles Intel AMT VNC semi-gracefully and only spams the kernel log with dozens of messages per second
  • ESXi 5.0 worked fine (if I remember correctly)

I have no idea what VMware is doing there. From all I can tell, out-of-band management like Intel AMT should be completely invisible to the OS.

Note that this is on a Sandy Bridge generation machine with an Intel C206 chipset and a Xeon E3-1225. The Q67 chipset is almost identical to the C206, so I expect it to occur there as well. Newer chipsets hopefully behave better, perhaps even newer firmware versions help.

Update November 2014: I just upgraded to the latest version, ESXi 5.5u2-2143827, and it’s working again. I still get the dmesg spam, but the PSODs are gone. These are the kernel messages I’m seeing now while connected via Intel AMT VNC:

2014-11-29T11:17:25.516Z cpu0:32796)WARNING: IOMMUIntel: 2493: IOMMU context entry dump for 0000:00:02.0 Ctx-Hi = 0x101 Ctx-Lo = 0x10ec22001
2014-11-29T11:17:25.516Z cpu0:32796)WARNING: IOMMU: 1652: IOMMU Fault detected for 0000:00:02.0 (unnamed) IOaddr: 0x5dc5aa000 Mask: 0xc Domain: 0x41089f1eb400
2014-11-29T11:17:25.516Z cpu0:32796)WARNING: IOMMUIntel: 2436:  DMAR Fault IOMMU Unit #0: R/W=R, Device 0000:00:02.0 Faulting addr = 0x5dc5aa000 Fault Reason = 0x0c -> Reserved fields set in PTE actively set for Read or Write.

So basically, Intel AMT VNC is now usable again.

Update August 2015: ESXi 6.0 still spams the logs, no change over ESXi 5.5.

OpenWRT on TP-Link TL-WDR3600: enabling Wifi channel 12+13 and higher power on 5 GHz

A few months ago I recommended the TP-Link TL-WDR3600 as an OpenWRT router. I did complain that it unnecessarily limits me to 50 mW in the 5 GHz. After I discovered that it also prevents me from using channel 12 and 13 (which are only available in Europe an Japan, but not in the US), I looked around for a solution.

The in-kernel regulatory database is not the issue. Running iw reg get on the router shows:

country DE:
 (2400 - 2483 @ 40), (N/A, 20)
 (5150 - 5250 @ 40), (N/A, 20), NO-OUTDOOR
 (5250 - 5350 @ 40), (N/A, 20), NO-OUTDOOR, DFS
 (5470 - 5725 @ 40), (N/A, 27), DFS

Some googling around reveals that the ath9k wireless chips have a bit in EEPROM that may be set to either US or worldwide. Apparently, there is a law in the US or an FCC regulation that requires all Wifi devices shipped to determine on the hardware level (in this case, in the driver) whether the desired frequency and power level is allowed.

Unfortunately, TP-Link simply sets the bit to US mode on all devices shipped worldwide. The stock firmware simply ignores it and offers frequency and power choice based on the selected country code. OpenWRT however (as would a stock Linux kernel) respects the bit and applies a logic AND over the selected regulatory domain and the US regulatory domain.

If you run iw phy phyX info, you can see the result:

* 2412 MHz [1] (20.0 dBm)
* 2417 MHz [2] (20.0 dBm)
* 2422 MHz [3] (20.0 dBm)
* 2427 MHz [4] (20.0 dBm)
* 2432 MHz [5] (20.0 dBm)
* 2437 MHz [6] (20.0 dBm)
* 2442 MHz [7] (20.0 dBm)
* 2447 MHz [8] (20.0 dBm)
* 2452 MHz [9] (20.0 dBm)
* 2457 MHz [10] (20.0 dBm)
* 2462 MHz [11] (20.0 dBm)
* 2467 MHz [12] (disabled)
* 2472 MHz [13] (disabled)
* 2484 MHz [14] (disabled)
* 5180 MHz [36] (17.0 dBm)
* 5200 MHz [40] (17.0 dBm)
* 5220 MHz [44] (17.0 dBm)
* 5240 MHz [48] (17.0 dBm)
* 5260 MHz [52] (20.0 dBm) (passive scanning, no IBSS, radar detection)
* 5280 MHz [56] (20.0 dBm) (passive scanning, no IBSS, radar detection)
* 5300 MHz [60] (20.0 dBm) (passive scanning, no IBSS, radar detection)
* 5320 MHz [64] (20.0 dBm) (passive scanning, no IBSS, radar detection)
* 5500 MHz [100] (20.0 dBm) (passive scanning, no IBSS, radar detection)
* 5520 MHz [104] (20.0 dBm) (passive scanning, no IBSS, radar detection)
* 5540 MHz [108] (20.0 dBm) (passive scanning, no IBSS, radar detection)
* 5560 MHz [112] (20.0 dBm) (passive scanning, no IBSS, radar detection)
* 5580 MHz [116] (20.0 dBm) (passive scanning, no IBSS, radar detection)
* 5600 MHz [120] (disabled)
* 5620 MHz [124] (disabled)
* 5640 MHz [128] (disabled)
* 5660 MHz [132] (20.0 dBm) (passive scanning, no IBSS, radar detection)
* 5680 MHz [136] (20.0 dBm) (passive scanning, no IBSS, radar detection)
* 5700 MHz [140] (20.0 dBm) (passive scanning, no IBSS, radar detection)
* 5745 MHz [149] (disabled)
* 5765 MHz [153] (disabled)
* 5785 MHz [157] (disabled)
* 5805 MHz [161] (disabled)
* 5825 MHz [165] (disabled)

Some earlier fixes that no longer work with the current OpenWRT involved editing the US regulatory domain in the userland regulatory database. Nowadays that’s part of the kernel itself and no longer easily possible.

Luckily, someone created a binary patch called reghack that replaces the in-driver US regulations with unrestricted ones by (as far as I can tell from the source code) permitting both 2400-2483 and  5140-5860 MHz with 40 MHz wide channels at up to 30 dBm and without any restriction flags. Only channel 14 still seems to be unavailable, but that’s not a big deal as it is only available in Japan for use with 802.11b (that 14-year-old protocol that did a maximum of 11 Mbit/s). Applying the patch is simple (it is downloadable in both source code and binary form), though I needed to perform a cold reboot after the reboot.

In Germany (country code DE), this gives me the following channel map:

* 2412 MHz [1] (20.0 dBm)
* 2417 MHz [2] (20.0 dBm)
* 2422 MHz [3] (20.0 dBm)
* 2427 MHz [4] (20.0 dBm)
* 2432 MHz [5] (20.0 dBm)
* 2437 MHz [6] (20.0 dBm)
* 2442 MHz [7] (20.0 dBm)
* 2447 MHz [8] (20.0 dBm)
* 2452 MHz [9] (20.0 dBm)
* 2457 MHz [10] (20.0 dBm)
* 2462 MHz [11] (20.0 dBm)
* 2467 MHz [12] (20.0 dBm)
* 2472 MHz [13] (20.0 dBm)
* 2484 MHz [14] (disabled)
* 5180 MHz [36] (20.0 dBm)
* 5200 MHz [40] (20.0 dBm)
* 5220 MHz [44] (20.0 dBm)
* 5240 MHz [48] (20.0 dBm)
* 5260 MHz [52] (20.0 dBm) (radar detection)
* 5280 MHz [56] (20.0 dBm) (radar detection)
* 5300 MHz [60] (20.0 dBm) (radar detection)
* 5320 MHz [64] (20.0 dBm) (radar detection)
* 5500 MHz [100] (27.0 dBm) (radar detection)
* 5520 MHz [104] (27.0 dBm) (radar detection)
* 5540 MHz [108] (27.0 dBm) (radar detection)
* 5560 MHz [112] (27.0 dBm) (radar detection)
* 5580 MHz [116] (27.0 dBm) (radar detection)
* 5600 MHz [120] (27.0 dBm) (radar detection)
* 5620 MHz [124] (27.0 dBm) (radar detection)
* 5640 MHz [128] (27.0 dBm) (radar detection)
* 5660 MHz [132] (27.0 dBm) (radar detection)
* 5680 MHz [136] (27.0 dBm) (radar detection)
* 5700 MHz [140] (27.0 dBm) (radar detection)
* 5745 MHz [149] (disabled)
* 5765 MHz [153] (disabled)
* 5785 MHz [157] (disabled)
* 5805 MHz [161] (disabled)
* 5825 MHz [165] (disabled)

Note that this is still not entirely what is permitted in Germany: 5150-5250 and 5250-5350 MHz may go up to 200 mW and 5470-5725 even up to 1 W. Since radar detection is not currently supported on the OpenWRT, the latter two ranges are not usable anyway. I’d have liked to turn up transmission power to 200 mW, but as it turns out, TP-Link saved a few cents on the 5 GHz power amplifier, which doesn’t even do more than 63 mW…

Warning: Before you change channels and transmission power on your Wifi devices, check with the regulation authority what is legally allowed in your location. The hardware is capable of things that can interfere with radar etc. and you should never set it to a country code other than your current location. Even then, you might be able to choose options that are not legal to operate.

For Germany, the Bundesnetzagentur has the official frequency allocation documents on their website: 2.4 GHz and 5 GHz. Other European countries should have similar authorities and similar allocations. If you’re in the US, don’t even bother applying these changes as the FCC does not permit anything beyond what OpenWRT is already capable by itself.

Using Intel AMT’s VNC server

Newer Intel Chipsets with vPro/Intel AMT, such as the Q57, Q67 and C206 (as long as they’re paired with a Core i5/i7 or Xeon with integrated graphics), have a feature called Remote KVM.

To use it, press Ctrl-P at the BIOS splash screen to get to the MEBx menu, set a password (minimum 8 characters, mixed case, numbers and special characters are enforced), configure the network settings (they can even match the OS’s IP address), enable Remote KVM and disable User Opt-In.

Next, download the Intel AMT SDK, extract the ZIP and open .\Windows\Intel_AMT\Bin\KVM\KVMControlApplication.exe . There, you can enable KVM as seen in the following screenshot:

KVM Status can either be set to “redirection ports” (meaning it will only be accessible to VNC clients that specifically support Intel AMT, such as RealVNC Viewer Plus or Intel’s KVM Console, the former of which costs $100, the latter of which constantly overlays a RealVNC logo on the screen), to “default port” (meaning it will be accessible on TCP port 5900 to any VNC client), or to “all ports” (which is the combination of both).
If you enable VNC access, you will also need to set an RFB Password. As I found out the hard way (Intel actually has it hidden in their documentation as well), it gets truncated at 8 characters and at the same time has the same security requirements as the general AMT password.
If you disabled User Opt-In in the MEBx menu, you can disable it here as well.

So that’s it, now you can use almost any VNC client you like (RealVNC and Chicken of the VNC work fine, while Apple Remote Desktop appears to cause the VNC server to freeze) and control the machine just as if you were sitting in front of it.
Two things I noticed: On my machine, the BIOS splash screen was not visible during a KVM connection (not even on a directly-attached screen), so to get to the BIOS I needed to blindly hit the corresponding key. Also, it is not possible to enter the MEBx menu during a KVM connection (probably for some obscure security reasons): if you hit the corresponding key, it immediately exits and continues normal bot; if you establish a KVM connection while in MEBx, you get disconnected immediately.

After about half an hour of playing with Intel AMT, I have to say it’s really cool. If you’re buying/building a home server, you should definitely consider getting a mainboard with Intel AMT 6.0 or later: You get server-grade remote management capabilities for a very small premium, which are very useful if you ever lock yourself out while remotely connected to the server.