Category Archives: Uncategorized

Wiring Fibre Channel for Arbitrated Loop

We have a small Fibre Channel SAN with three servers, a switch and a dual-controller RAID enclosure. With only a single switch, we obviously couldn’t connect all servers redundantly to the RAID system. That meant, for example, that firmware updates to it could only be applied after shutting down the servers. Buying a second switch was hard to justify for this simple setup, so we decided to hook the switch up to the first RAID controller and wire a loop off the second RAID controller. Each server would have one port connected to the switch and another one to the loop.

Back in the old days, Fibre Channel hubs existed for exactly this purpose, but nowadays you simply can’t get them anymore. However, in a redundant setup, you don’t need a hub, you can simply run cables appropriately, i.e. in a daisy-chain fashion. The only downside of not having a hub, the loop going down if one server goes down, is irrelevant here because you have a second path via the switch. For technical details on the loop topology, you can have a look at documentation available from EMC.

To wire your servers and RAID controllers in a fashion resembling a hub (only without the automatic bypass if a server goes down), you need simplex patch cables. These consist of a single fibre (instead of two, like you are used to) and have a connector that looks like half of a regular LC connector. You can get them in the same qualities as regular (duplex) patch cables and in single- and multi-mode as needed. They look like this:

These cables are somewhat exotic, so your usual cable dealer might not have them, but they exist and are available from specialized fiber cable dealers. As you are wiring in a daisy-chain fashion, you need one simplex cable per node you want to connect.

Once you have the necessary cables, you wire everything by connecting a cable from the output side of the FC port on the first device and connecting it to the input side of the FC port on the second device. The second device’s output side connects to the third device’s input side and so on, until you have made a full loop by connecting the last device’s output side to the first device’s input side, like this:

How can you tell the output side from the input side? Some transceivers have little arrows or have labels “TX” (output) and “RX” (input). If yours don’t, you can recognize the output side by the red laser light coming from it. To protect your eyes, never look directly into the laser light. Never connect two output sides together, otherwise you may damage the laser diodes in both of them. Therefore, be careful and always double-check.

That’s it, you’re done. All servers on the loop should immediately see the LUNs exported to the by the RAID controller.

In my setup however, further troubleshooting was required. The loop simply did not come up. As it turns out, someone had set the ports on one FC HBA to point-to-point-only and 2 Gbit/s mode. After switching both these settings to their default automatic mode, the loop went up and data started flowing. My HBAs are QLogic QLE2462 (4 Gbit/s generation) and QLE2562 (8 Gbit/s generation), and they automatically negotiated the fastest common speed they could handle, which is 4 Gbit/s. Configuring these parameters on the HBA usually requires hitting a key at the HBA’s pre-boot screen to enter the configuration menu and doing it there, or via vendor-specific software. I didn’t have access to the pre-boot configuration menu and was running VMware ESXi 6.0 on the servers. The QConvergeConsole for my QLogic adapters luckily is also available for VMware. It is not as easy to install as on Windows or Linux, unfortunately. You first install a CIM provider via the command line on the ESXi host, reboot the host and then install a plugin into VMware vCenter server.

OpenMPI and SLURM CPU pinning

The SLURM scheduling and queueing system pins processes to CPUs. This worked great with OpenMPI 1.6, but OpenMPI 1.8 introduced its own pinning policy. As a result, our jobs ended up running very slowly because all jobs that used less than a whole machine ended up getting pinned to cores 0 through n, where n is the number of cores requested by a job.

Fixing this is pretty simple: just disable OpenMPI’s pinning so that the MPI processes inherit the scheduler’s pinning. To do that, add the following to your job script:

export OMPI_MCA_hwloc_base_binding_policy=none

Obviously, as a more permanent and more fool-proof solution, set that setting in the OpenMPI config file on all of your compute nodes, have the scheduler set that environment variable, or put it into your module file for OpenMPI.

Tracking DHL Freight packages

I recently got a shipment notification with a tracking number starting with JND and followed by 17 digits, half of them zero. After searching around for a while, I figured it was shipped by DHL Freight. It’s kind of hard to find the tracking page for that and it’s not supported by Delivery Status, so here’s the link: It’s hidden on the DHL web site; you can find it by going to the Logistics tab in the tracking section and clicking “Road and rail”.

Ich habe kürzlich eine Versandbenachrichtigung mit einer Trackingnummer erhalten, die mit JND anfängt. Wie sich herausstellt, gehören solche Trackingnummern zu DHL Freight. Die Tracking-Seite ist etwas schwierig zu finden und Delivery Status unterstützt DHL Freight auch nicht. Der Direktlink ist und er ist recht versteckt auf der DHL-Webseite: Man muss auf das Logistik-Tab gehen und dort auf “Land- und Schienenverkehre” klicken.

Active PA speaker systems

During my high school time, I did a lot of work as a sound technician and lighting designer at all kinds of events, both as a volunteer at school and outside of school. Recently someone from the school told me that they were looking at buying a new portable speaker system, both to replace their old/broken/underpowered one and because they were paying more than a thousand euros in rental fees for additional speakers every year. They asked me if I could help them choose one. We quickly reached the conclusion that active speakers were the way to go because of their flexibility and because they would often be operated by people who didn’t know a lot about all the technical stuff.

We ended up narrowing it down to three candidates: the Yamaha DSR series, the JBL PRX600 series, and the QSC KW series. These are the top-of-the-line active systems the largest and most reputable speaker manufacturers have to offer, as of early 2012. The next step was to find a place where we could listen to all three and compare them. Thomann, the largest online shop for musical instruments and PA equipment in Europe, where we have been buying sound equipment for years, has a huge store and showroom in a small town in northern Bavaria. They ordered and set up all these speakers for us and let us listen to them for more than an hour. If the friendly sales guy ever grew tired of listening to our test songs like “He’s A Pirate” by Klaus Badelt or “Man in the Mirror” by Michael Jackson over and over again, he certainly didn’t show it.

We almost immediately ruled out the QSC: We had the QSC KW 153 three-way 15″ top coupled with a QSC KW 181 18″ subwoofer set up, but the mids and highs just sounded muddy.

Some other speakers we temporarily had in the test were some JBL Eon (by accident), which just sounded cheap compared to the others, and some RCF Art, which had crisp and powerful base, but not exactly outstanding highs.

Now we only had the JBL PRX 615 two-way 15″ top coupled with the JBL PRX618-XLF 18″ subwoofer and the Yamaha DSR 115 two-way 15″ top coupled with the Yamaha DSR 118W 18″ subwoofer left over. We tested and compared them for almost an hour: sometimes we tended towards the JBLs, other times we liked the Yamahas more.

The JBLs sounded very smooth (if you want to be mean, you could call them a tiny bit muddy) and their base stretched down to 30 Hz. On the other hand, the Yamahas had super-clear mids and highs and very precise and crisp base. After quite some discussion, we decided to go with the Yamahas. Another advantage was their significantly lower price and their more advanced DSP circuitry to protect the speakers.

They got delivered a few days ago and so far we’re really happy with them. They sound amazing: Perfect for the school’s numerous music performances. And they are powerful: Perfect for events like dances and parties.

If you’re looking for a set of high-quality portable speakers for a school, church, band or DJ, the Yamaha DSR series is most likely your best choice. They sound great, have lots of power and are well worth their money.

If you just use them for speech, a set of Yamaha DSR 112 would probably be a fine choice (I didn’t test the DSR 112, but assume they’re as good as the DSR 115 with a little less low-mids). If you’re using them for a band, go for a set of DSR 115, and if you have drums, base or anything else below 120 Hz, definitely get a pair of DSR 118W subwoofers along with them. Same goes for DJ and party use: a pair each of DSR 115 and DSR 118W should suffice to bring high-decibel, high-quality sound to a few hundred audience members.

If you have experiences with the Yamaha DSR (or the recently-released smaller DXR and DSW series), or have found other speakers in the same price range that sound better, please feel free to share them in the comments.