p.o.d logo


 Alexander Færøy
 Anders Ossowicki
 Andreas Bach Aaen
 Anton Berezin
 Bryan Østergaard
 Carsten Pedersen
 Christian Jørgensen
 Christian Sejersen
 Christina Rudkjøbing
 Dan Leinir Turthra Jensen
 David Zeuthen
 Erwin Lansing
 Flemming Jacobsen
 Frederik S. Olesen
 Georg Sluyterman
 Henrik Brix Andersen
 Henrik Lund Kramshøj
 Henrik Tudborg
 Jesper Dangaard Brouer
 Jesper Jarlskov
 Jesper Krogh
 Jesper Louis Andersen
 Jesper Nyerup
 Josef Assad
 Kenneth Christiansen
 Kenneth Geisshirt
 Klavs Klavsen
 Kristian Høgsberg
 Kristian Nielsen
 Lars Knudsen
 Lars Sommer
 Lars Sommer
 Leif Lodahl
 Mads Toftum
 Martin Hansen
 Martin Pihl
 Martin Schlander
 Martin von Haller Grønbæk
 Martin von Haller Grønbæk
 Niels Kjøller Hansen
 Nikolaj Hald Nielsen
 Peter Hansteen
 Peter Larsen
 Peter Makholm
 Peter Rude
 Peter Toft
 Phil Regnauld
 Poul-Henning Kamp
 Sune Kloppenborg Jeppesen
 Sune Vuorela
 Søren Bredlund Caspersen
 Søren Hansen
 Søren Hansen
 Søren Sandmann
 Søren Straarup
 Thilo Bangert
 Thomas Alexander Frederiksen
 Thomas H.P. Andersen
 Thor Dekov Buur

Sidst opdateret:
September 20, 2014, 08:01 UTC

Dette er folks egne meninger og har ikke nødvendigvis nogen forbindelse til hvad danske opensource-foreninger mener.

Kom på:
Kontakt planet@opensource.dk hvis du føler du vil være her.

Inkluder venligst URL for det feed du ønsker optaget, samt en redegørelse for opfyldelsen af nedenstående krav.

  • Du er aktivt involveret i free/open source software (udvikling, foreninger e.l.)
  • Du skriver på skandinavisk og/eller på engelsk
  • Du har en tilknytning til skandinavien
  • Du skriver en gang i mellem om noget relevant for free/open source software-verdenen

Powered by:
September 14, 2014
Peter Toft a.k.a. pto
Peter Toft Jeg har for nylig købt en Gigabyte Brix computer med en Intel Celeron N2807. Maskinen fik en 4GB RAM klods og en SSD disk i og da den leveres uden operativsystem, så måtte jeg i gang med at installere. Jeg smed en Ubuntu 14.04 på maskinen med XBMC for at se hvordan den arter sig. Som sådan virke...

Leave a comment

September 13, 2014
p { margin-bottom: 0.25cm; line-height: 120%; } This proposal has been said many times over the last couple of years and lately repeated by Daniel Brunner, head of the IT department of Switzerland's Federal Supreme Court.https://joinup.ec.europa.eu/community/osor/news/open-and-libre-office-projects-should-reunite. And from the first point of view I can only agree. There is no reason what
September 12, 2014
Poul-Henning Kamp a.k.a. phk
CSC sagen (September 12, 2014, 15:37 UTC)
Poul-Henning Kamp Jeg får en dårligere og dårligere smag i munden over den der CSC sag. Lad det være sagt med det samme at jeg slet ikke tager stilling til skyldsspørgsmålet, og at min opfattelse er at både de anklagede, rigspolitiet, anklagemyndigheden og CSC er nogle inkompetente klaphatte -- alle til hobe. Me...

Leave a comment

The tool "netperf-wrapper" (by +Toke Høiland-Jørgensen <toke(at)toke.dk>) is very useful for repeating network measurements, that involves running multiple concurrent instances of testing tools (primarily netperf, iperf and ping, but also tools like d-itg and http-getter).

The tools is best known in the bufferbloat community for it's test Realtime Response Under Load (RRUL), but the netperf-wrapper tool has other tests that I find useful.
Core software dependencies are recent versions of netperf, python, matplotlib and fping (optional are d-itg and http_runner).

Dependency issues on RHEL6

First dependencies are solved easily by installing "python-matplotlib":
 $ sudo yum install -y python-matplotlib python-pip

The software dependencies turned out to be a challenge on my RHEL6 box.

The "ping" program is too old to support option "-D" (prints timestamp before each-line).  Work-around is to install "fping", which I choose to do from "rpmforge":

Commands needed for install "fping":
 # rpm -Uvh http://packages.sw.be/rpmforge-release/rpmforge-release-0.5.2-2.el6.rf.x86_64.rpm
 # yum install -y fping

The "netperf" tool itself (on RHEL6) were not compiled with configure option "--enable-demo=yes" which is needed to get timestamp and continuous result output during a test-run.

Thus, I needed to recompile "netperf" manually:

Install netperf-wrapper

Installation is quite simple, once the dependencies have been meet:

  • git clone https://github.com/tohojo/netperf-wrapper.git
  • cd netperf-wrapper
  • sudo python2 setup.py install

GUI mode

There is a nice GUI mode for investigating and comparing results, started by:
 $ netperf-wrapper --gui

This depend on matplotlib with Qt4 (and PyQt4), which unfortunately were not available for RHEL6. Fortunately there were a software package for this on Fedora, named "python-matplotlib-qt4".

For GUI mode netperf-wrapper needs: matplotlib with Qt4
 $ sudo yum install -y python-matplotlib-qt4 PyQt4

Thus, the workflow is to run the tests on my RHEL6 machines, and analyze the result files on my Fedora laptop.

Using the tool

The same tool "netperf-wrapper" is both used for running the test, and later analyzing the result.

Listing the tests available:
 $ netperf-wrapper --list-tests

For listing which plots are available for a given test e.g. "rrul":
 $ netperf-wrapper --list-plots rrul

Before running a test towards a target system, remember to start the "netserver" daemon process on the target host (just run command "netserver" nothing else).

Start a test-run towards e.g. IP with test rrul
 $ netperf-wrapper -H -t my_title rrul

It is recommend using the option "-t" to give your test a title, which makes is easier to distinguish when comparing two or more test files in e.g. the GUI tool.

The results of the test-run will be stored in a compressed json formatted text file, with the naming convention: rrul-2014-MM-DDTHHMMSS.milisec.json.gz

To view the result, without the GUI, run:
 $ netperf-wrapper -i rrul_prio-2014-09-10T125650.993908.json.gz -f plot
Or e.g. selecting a specific plot like "ping_cdf"
 $ netperf-wrapper -i rrul_prio-2014-09-10T125650.993908.json.gz -f plot -p ping_cdf

netperf-wrapper can also output numeric data suitable for plotting in org-mode or .csv (spreadsheet) format, but I didn't play with those options.

Updates: A release 0.7.0 of netperf-wrapper is pending

Extra: On bufferbloat
Interested in more about bufferbloat?

Too few people are linking to the best talk explaining bufferbloat and how it's solved by Van Jacobson (slides).  The video quality is unfortunately not very good.

I've used some of Van's point in my own talk about bufferbloat: Beyond the existences of Bufferbloat, have we found the cure? (slides)
September 11, 2014
Network setup for accurate nanosec measurements (September 11, 2014, 12:37 UTC)
As I described in my previous blogpost, I'm leveraging the PPS measurements to deduct the nanosec improvements I'm making to the code.

One problem with using this on the nanosec scale is the accuracy of your measurements, which depend on the accuracy of the hardware your are using.

Modern systems have power-saving and turbo-boosting features build into the CPUs.  And Hyper-Threading technology that allows one CPU core to appear as two CPUs, by sharing ALUs etc.

While establish an accurate baseline for some upstream measurements (subj: Get rid of ndo_xmit_flush / commit 0b725a2ca61). I was starting to see too much variation in my trafgen measurements.

I created a rather-large oneliner, that I have converted into a script here: https://github.com/netoptimizer/network-testing/blob/master/bin/mon-ifpps
Which allowed me to get a picture of the accuracy of my measurements, and they are not accurate enough. (For more real stats like std-dev consider running these measurements through Rusty Russell's tool

My findings:

  1. Disable all C states and P states.
  2. Disabling Hyper-Threading and power-management in BIOS helped the accuracy
  3. 10Gbit/s ixgbe ring-buffer cleanup interval also influenced accuracy

Reading +Jeremy Eder's blog post. It seems the best method for disabling these C and P states, and
keeping all CPUs in C0/C1 state is doing:

 # tuned-adm profile latency-performance

I found the most stable ring-buffer cleanup interval for the ixgbe driver were 30 usecs. Configured by

 # ethtool -C eth5 rx-usecs 30

Besides these tunings: my blogpost on "Basic tuning for network overload testing" should still be followed.
Generally I've started using the "profile latency-performance", but unless I need to measure some specific code change, I'm still using the ixgbe's default "dynamic" ring-buffer cleanup interval.

Details about the "ethtool -C" tuning is avail in blogpost "Pktgen for network overload testing".
Why I'm using Packet Per Second (PPS) tests for measuring the improvement in performance (of the Linux Kernel network stack).  Many people (e.g. other kernel developer) does not understand why I'm using PPS measurements, this blogpost explains why.

The basic problem of using large MTU (usually 1500 bytes) size packets, is that the transmission delay itself, is enough to hide any improvement I'm making (e.g. a faster lookup function).

Transmission delay 1514 bytes (+20 bytes for Ethernet overhead) at 10Gbit/s is 1227 nanosec:

  • ((bytes+wireoverhead)*8) / 10 Gbits = time-unit
  • ((1500+14+20)*8)/((10000*10^6))*10^9 = 1227.20 ns

This means, if the network stack can generate (alloc/fill/free) a 1500 byte packet faster than every 1227ns, then it can utilize the bandwidth of the 10Gbit/s link fully.  And yes, we can already do so. Thus, with 1500 bytes frame any stack performance improvement, will only be measurable with by a lower CPU utilization.

Let face it; the kernel have been optimized heavily for the last 20 years.  Thus, the improvements we are able to come up with, is going to be on the nanosec scale.
For example I've found a faster way to clear the SKB, which saves 7 nanosec.  Being able to measure this performance improvement were essential while developing this faster clearing.

Lets assume, the stack cost (alloc/fill/syscall/free) is 1200ns (thus faster than 1227ns), then a 7ns improvement will only be 0.58%, which I can only measure as a lower CPU utilization (as bandwidth limit have been reached), which in practice cannot be measured accurately enough.

By lowering the packet size, the transmission delay the stack cost (alloc/fill/syscall/free) can "hide-behind" is reduced. With the smallest packet size of 64 bytes, this is significantly reduced, to:

  • ((64+20)*8)/((10000*10^6))*10^9 = 67.2ns

This basically exposes stacks cost, as its current cost is larger than 67.2ns.  This can be used for getting some measurements that allow us to actually measure the improvement of the code changes we are making, even-though this "only" translates into reduced CPU usage with big frames (which translates into more processer time for your application).

In packet per sec (pps) this correspond to approx 14.8Mpps:

  • 1sec/67.2ns =>  1/(67.2/10^9) = 14,880,952 pps
  • or directly from the packet size as:
  • 10Gbit / (bytes*8) = (10000*10^6)/((64+20)*8) = 14,880,952 pps

Measuring packet per sec (PPS) instead of bandwidth, have another advantage.  Instead of just comparing how many PPS improvement is seen, then instead translate the PPS into nanosec (between packets).
Comparing nanosec used before and after, will show us the nanosec saved by the given code change.

See, how I used it in this and  this commit to document the actually improvement of the changes I made.

Update: For deducting the nanosec saved by a given code change, to be valid, usually requires isolating your test to utilize a single CPU.

Lets use the 14.8Mpps as an example of howto translate PPS to nanosec:

  • 1sec / pps => (1/14880952*10^9) = 67.2ns

Extra: Just for the calculation exercise.

  • How many packets per sec does 1227.20 ns correspond to:
    • 1sec/1227.2ns =>  1/(1227.2/10^9) = 814,863 pps
  • Can also be calculated directly from the packet size as:
    • 10Gbit / (bytes*8) = (10000*10^6)/((1514+20)*8) = 814,863 pps

September 02, 2014
Poul-Henning Kamp a.k.a. phk
NSA's porno-business. (September 02, 2014, 20:30 UTC)
Poul-Henning Kamp I de første afsløringer fra Snowden kom det frem at alle de store "cloud" firmaer var "trojaned" af NSA. I et stort interview med The Guardian for nylig nævner Edward Snowden at adgang til nøgenbilleder var en slags "personalegode" i NSA. Nogle uger senere dukker en masse kendis-nøgenbilleder o...

Leave a comment

August 31, 2014
Peter Toft a.k.a. pto
Elementary OS - dit næste Linux-valg? (August 31, 2014, 20:52 UTC)
Peter Toft I dette blogindlæg ser jeg nærmere på et interessant valg af Linux-variant: Elementary OS, der tilbyder en enkel men lækker brugergrænseflade. Brugergrænsefladen på min computer betyder meget for de fleste af os (og ikke et ord om Windows 8.x) . Indenfor Linux er der Ubuntus Unity, som mange er ...

Leave a comment

August 26, 2014
Poul-Henning Kamp a.k.a. phk
NemID2: Gør det nu rigtigt 5/5 (August 26, 2014, 09:46 UTC)
Poul-Henning Kamp Følgetonen fortsætter: Første afsnit var om gammel historie. Andet afsnit om CPRs historie. Tredje afsnit om Digtal Signatur. Fjerde afsnit om NEMID. Spørgsmålet er hvad vi, Danmark, skal gøre når kontrakten om den nuværende NemID snart udløber. Min holdning er at vi skal gribe det helt and...

Leave a comment

August 25, 2014
Peter Makholm a.k.a. brother
Peter Makholm
Gennem tiden har jeg flere gange implementeret forskellige former for autentificering. Nogle gange har jeg været bundet af forskellige protokol-beslutninger og andre gange har jeg været mere frit stillet. Derfor har jeg også haft mulighed for at samle mig en række ideer om hvordan autentificerin...
August 23, 2014
Anton Berezin a.k.a. Grrrr
YAPC::Europe 2014, day 2 (August 23, 2014, 14:23 UTC)

Ignat Ignatov talked about physical formulas. When I was planning to attend this talk, I thought it is going to be some sort of symbolic formulas computation, possibly with an analysis of dimensions of the physical quantities.
However, despite my (a bit long in the tooth) background in physics, I did not understand a word of it. Apparently, some sort of unification of physical formulas, not entirely unlike the periodic table in chemistry, was presented, with almost no comprehensible details and with scary words like co-homology and algebraic topology. The fact that half of the slides were in Russian, while irrelevant for me personally, probably did not help matters for the majority of the people in the audience. I did not expect any questions at the end of the talk, but there were at least two, so I was probably wrong about general level of understanding in the audience.

Laurent Dami talked about SQL::Abstract::FromQuery. He presented a query form of the Request Tracker and said that it is too complex - a premise many would agree with. The conclusion was that some more natural way to allow the user to specify complex queries is needed. Surprizingly, the answer to that was to use a formal grammar and make the user adhere to it. To me this sounds weird, but if one can find a non-empty set of users that would tolerate this, it may just work.

Denis Banovic talked about Docker, a virtualization container. I did not know much about Docker until this point, so it was useful to have someone to explain it to me.

The next talk was long, 50 minutes (as opposed to a somewhat standard for this conference 20 minutes) Peter "ribasushi" Rabbitson presented a crash-course in SQL syntax and concepts. It looked like a beginner-level introduction to SQL, but it became better and better as it progressed. I even learned a thing or two myself. ribasushi has a way of explaining rather complicated things concisely, understandably, and memorizably at the same time. Excellent talk.

Then there was a customary Subway sandwiches lunch.

Naim Shafiyev talked about network infrastructure automatization. Since this is closely related to what I do at my day job, I paid considerable attention to what he had to say. I did not hear anything new, but hopefuly the rest of the audience found the talk more useful. It did inspire me to submit a lightning talk though.

osfameron talked about immutable data structures in Perl and how to clone them with modifications, while making sure that the code does not look too ugly. Pretty standard stuff for functional languages, but pretty unusual in the land of Perl. The presentation was lively, with a lot of funny pictures and Donald duck examples.

The coffee break was followed by another session of lightning talks, preceeded by a give-away of a number of free books for the first-time YAPC attendees. Among the talks I remembered were SQLite virtual tables support in Perl by Laurent Dami, web-based database table editor by Simun Kodzoman, LeoNerd's presentation about XMPP replacement called Matrix, a Turing-complete (even if obfuscated) templating system by Jean-Baptiste Mazon of Sophia (sp!), and annoucements of Nordic Perl Workshop 2014 (Helsinki, November) and Nordic Perl Workshop 2015 (Oslo, May).

Again, I did not go to the end-of-the-day keynote.

As a side note, the wireless seemed to be substantially more flaky than yesterday, which has affected at least some lightning talk presenters.

August 22, 2014
Anton Berezin a.k.a. Grrrr
YAPC::Europe 2014, day 1 (August 22, 2014, 20:54 UTC)

When I came to the venue 15 minutes before the official start of the registration, people at the registration desk were busily cutting sheets of paper into attendees' badges. Finding my badge turned out to be a tad not trivial.

This conference is somewhat unusual not only because it is conducted over the weekend instead of in the middle of the week, but also because the keynotes for every day are pushed till the end, even after the daily lightning talks session.

The welcome talk from Marian was about practical things such as rooms locations, dinner, lunches, transportations and so on. Then I went on stage to declare the location of YAPC::Europe 2015 (which is Granada, Spain by the way). After that Jose Luis Martinez from Barcelona.pm did a short presentation of YAPC in Granada, and Diego Kuperman gave a little present from Granada to Sofia.

Mihai Pop of Cluj.pm presented a talk called "Perl Secret". It was basically a 20-minutes version of BooK's lightning talk about Perl secret operators, somewhat duluted by interspersing references to minions. It was entertaining.

The great Mark Overmeer talked about translation with context. He went beyond the usual example of multiple variants of plural values in some languages, and talked about solving localization problems related to gender and so on. The module solving these problems is Log::Report::Translate::Context. As always, great attention to details from Mark.

After lunch (sandwiches from Subway), Alex Balhatchet of Nestoria presented hurdles of geocoding, with solutions. I and my co-workers had encountered similar problems on a far smaller scale, so I could understand the pains, and had a great interest in hearing about the solutions.

Then I attended a very inspiring talk by Max Maischein from Frankfurt about using Perl as a DNLA remote and as a DNLA media server. I immediately felt the urge to play with the code he published and try to adapt it to my own TV at home. There was even a live demo of using DNLA to stream to Max's laptop a live stream of the talk provided by the conference organizers. And it even worked, mostly.

Ervin Ruci talked more about geocoding — this talk was partially touching the same problems Alex Balhatchet was talking about. Unfortunately, it was substantially less detailed, so I was somewhat underwhelmed by it. The presenter mentioned cool things like dealing with fuzzyness of the input data using hidden Markov models, but did not expand on them.

StrayTaoist described how to access raw data from space telescopes using (of course) Perl. Very lively talk. There was a lot of austronomy porn in here.

Luboŝ Kolouch from Czech Republic talked about automotive logistics, and how open source solutions work where proprietory solutions do not. The software needs to be reliable enough to make sure that it takes only 1.5 hours between the part order and its physical delivery to the factory.

After coffee break with more mingling the inimitable R Geoffrey Avery choir-mastered an hour of lightning talks. Most talks were somewhat "serious" today; I hope we see more "fun" ones in the next coming days.

Unfortunately, I missed the first keynote of the conference from Curtis "Ovid" Poe, so cannot really say anything about it.

Finally, we went to Restaurant Lebed for the conference dinner. The location is superb, there is a great view over a lake. The food was great, too. We also got to enjoy some ethnic Bulgarian music and dancing, not too much, and not too little.

Lots of cheers to Marian and the team of volunteers for organizing what so far turns out to be a great conference.

Poul-Henning Kamp a.k.a. phk
NEMID hvordan 4/mange (August 22, 2014, 12:37 UTC)
Poul-Henning Kamp Følgetonen fortsætter: Første afsnit var om gammel historie. Andet afsnit om CPRs historie. Tredje afsnit om Digtal Signatur. Nu er vi nået til NemID som vi kender den idag og hvorfor den endte som den gjorde. Lad os tage det gode først: Man indså at der skulle en eller anden tofaktor authe...

Leave a comment

August 19, 2014
Poul-Henning Kamp a.k.a. phk
Digital Signatur: NemID prototypen 3/mange (August 19, 2014, 16:15 UTC)
Poul-Henning Kamp Første afsnit var om gammel historie. Andet afsnit om CPRs historie. Nu er turen kommet til den Digitale Signatur der var den umiddelbare forløber for NEMID som vi kender det. For nu at citere en der arbejdede rigtig meget med den Digitale Signatur, så var der kun fire problemer med den: Sign...

Leave a comment

August 18, 2014
Jesper Nyerup a.k.a. nyerup
Mirroring Ceph (August 18, 2014, 07:33 UTC)
Jesper Nyerup

I’m glad to announce that One.com‘s public mirror service has begun mirroring Ceph‘s download section. Ceph is a distributed object store and file system, which scales elegantly and has excellent fault tolerance. Ceph has official mirrors in the Western US and the Netherlands, and a handful of community driven mirrors all over the world — now including this one in Denmark, well connected in Northern Europe. We welcome anyone using it to suit their needs. The Ceph mirror is available over HTTP here, and is also available over Rsync and FTP.

One.com run their mirror service both for operational independence and to be able to give something back to the open source software community. The service mirrors a number of open source projects, and more are added frequently. I’m lucky to be part of the team of mirror maintainers, and we’d love to hear from you if you have questions or ideas for the service.

August 17, 2014
Peter Toft a.k.a. pto
Peter Toft Hvor det forrige blog-indlæg omhandlede Google maps, så er det også interessant at se hvad Google mener jeg har interesser i. På http://www.google.com/settings/ads er der lidt interessant læsning for mig (I har nok tilsvarende) At der i listen over hvad Google mener jeg interesserer mig for s...

Leave a comment

Peter Toft Flere af mine venner på Facebook postede tilsammen en del Google-information, som er værd at samle op på her. Jeg har delt de to historier i to blog-indlæg - dette og et andet (tryk her for at læse med). Jeg bruger Google maps ret så ofte, især til at undgå at køre mod et sted hvor man kører for...

Leave a comment

Poul-Henning Kamp a.k.a. phk
CPR: NemIDs bedstefar 2/mange (August 17, 2014, 10:03 UTC)
Poul-Henning Kamp I forrige blogindlæg ridsede jeg det historiske forhold imellem person-identitet og statsmagt op. Som vi så var stort set alle identifikationssystemer baseret på hvad man idag vil kalde "ihændehaverbeviser", selvauthenticerende dokumenter som den identificerede person bare kunne forevise og som ...

Leave a comment

August 15, 2014
Peter Makholm a.k.a. brother
Version2.dk: Digital Post, hvad nu? (August 15, 2014, 11:53 UTC)
Peter Makholm
Jeg har længe luret på hvad jeg gør når det offentlige den 1. november gennemtvinger brugen af Digital Post. At skulle læse henvendelser fra det offentlige på et websted der ikke passer ind i min normale rutine er efter min mening uhensigtsmæssigt. Det er især taget i betragtning af hvor lidt pos...
Martin Schlander a.k.a. cb400f
Four great technological advances (August 15, 2014, 08:28 UTC)
Martin Schlander

#1: openSUSE Factory Rolling Release Distribution

Over the course of the last several months a lot of changes were made to the development process for openSUSE Factory. Meaning it’s no longer a highly experimental testing dump, but it’s now a viable rolling release distribution in its own right. You can read all about the details here. I installed openSUSE Factory in a virtual machine yesterday and it seems to run pretty great. Of course to really judge a rolling release distribution you need to run it for a sustained period of time.

No rolling release distribution will ever be my preferred day-to-day operating system, but nevertheless I’m pretty excited about the “new” openSUSE Factory. I think the changes will enable version whores and bleeding edge explorers to finally have a truly symbiotic relationship with the users who value productivity and predictability in their PC operating system.


#2: KDE Frameworks 5 and Plasma 5

Since I was already testing openSUSE Factory it was a great opportunity to finally get my feet wet with the new KDE Frameworks 5 and Qt5 based KDE Plasma 5 workspace, initially released about a month ago. Obviously it’s still lacking some features and polish, but it’s already usable for forgiving users who know what they’re doing and showing great promise.


#3: 4G on the Jolla

My provider enabled 4G on my subscription and offered to send me a new SIM Card gratis. So now my Jolla is sporting 4G. Unfortunately it only took about 5-10 minutes of speed testing (peaking at 12 MB/s, averaging about 10 MB/s) to use all my available bandwidth for the month, so for the rest of August I’ve been speed dropped to 64 Kbps, but hey, it’s still 4G!


#4: Richard Stallman presenting with a slideshow

Who’d have ever thought they’d see the day that Stallman would do a presentation with accompanying slides? Well it happened, and I think this great use of slides helps him communicate more effectively. Watch the video and judge for yourselves (27 MB, 13 minutes).


August 14, 2014
Poul-Henning Kamp a.k.a. phk
Zoom ud fra NemID 1/mange (August 14, 2014, 17:36 UTC)
Poul-Henning Kamp Jeg har været til visionarium idag hos V2, fire fra digitaliseringsstyrelsen, fire udefra, armlægning efter detroit reglerne. Sådan cirka. Det var en fornøjelse uden lige, som det altid er at være i intelligente passionerede menneskers selskab. V2's journalister skriver om hvad der gik for sig...

Leave a comment

August 12, 2014
Poul-Henning Kamp a.k.a. phk
NemID2: Idestorm (August 12, 2014, 07:54 UTC)
Poul-Henning Kamp Antag at NemID2 bliver lavet rigtigt. Lad os ikke hænge os i detaljerne af hvad "rigtigt" indebærer med kryptografi og lovgivning, bare antag et øjeblik at det er gjort rigtigt -- Helt Rigtigt. Hvad kan du bruge din NemID2 til, som NemID1 ikke kan bruges til ? phk

Leave a comment

August 11, 2014
Peter Toft a.k.a. pto
Jeg er bekymret - er mine data sikre? (August 11, 2014, 05:50 UTC)
Peter Toft Jeg er bekymret. Sommeren er bl.a. gået med at få læst op på Snowden-sagen, og mange andre nyhedshistorier om "privacy". Vi skal nok til at gentænke meget af de tjenester vi bare tager for givet på internettet. (billedet er fra wikipedia) Jeg er bekymret. Indenfor det sidste par år har Faceb...

Leave a comment

August 08, 2014
Sune Vuorela a.k.a. pusling
Fun and joy with .bat files (August 08, 2014, 07:16 UTC)
Sune Vuorela

Occasionally, one gets in touch with kind of ‘foreign’ technologies and needs to get stuff working anyways.

Recently, I had to do some various hacking with and around .bat files. Bat files are a kind of script files for Microsoft Windows.

Calling external commands

Imagine need to call some other command, let’s say git diff. So from a cmd thing, you would write

git diff

similar to writing shell scripts on unixes. But there is a catch. If the thing you want to call is another bat-script, just calling it ensures it ‘replaces’ the current script and never returns. So you need

call git diff

if the command you want to run is a bat file and you want to return to your script.

Calling an external helper next to your script
If you for some reason needs to call some external helper placed next to your script, there is a helpful thing to do that as well. Imagine your helper is called helper.bat

call %~dp0helper.bat

is the very self-explanatory way of doing that.

Stopping execution of your script

If you somehow encounter some condition in your script that requires you to stop your script, there is a command ‘exit’ handy. It even takes a argument for what error code there is.

exit 2

stops your script with return code 2. But it also have the nice added feature that if you do it in a script you run by hand in a terminal, it also exits the terminal.

Luckily there is also a fix for that:

exit /b 2

and it doesn’t exit your interactive terminal, and it sets the %ERRORLEVEL% variable to the exit code.

Fortunately, the fun doesn’t stop here.

If the script is run non-interactively, exit /b doesn’t set the exit code for for example perl’s system() call. You need to use exit without /b for that. So now you need two scripts. one for “interactive” use that calls exit /b and a similar one using exit for use by other apps/scripts.

Or, we can combine some of our knowledge and add a extra layer of indirection.

  • write your script for interactive use (with exit /b) and let’s call it script.bat
  • create a simple wrapper script
    call %~dp0script.bat
    exit %ERRORLEVEL%
  • call the wrapper for non-interactive use
  • and then success.

    Oh. and on a unrelated note. Windows can’t schedule tasks for users that aren’t logged in and don’t have a password set. The response “Access Denied” is the only clue given.

    Leave a comment

    August 07, 2014
    Poul-Henning Kamp a.k.a. phk
    Lyt til Dan Geer (August 07, 2014, 10:54 UTC)
    Poul-Henning Kamp For nogen tid siden skrev Dan Geer om produktansvar og jeg tillod mig at droppe ham en email med en link til det jeg selv havde skrevet i ACM Queue om emnet. Det resulterede i at jeg blev medforfatter på en artikel som dukker op i en IEEE publikation inden længe, og som langt hen ad vejen er gru...

    Leave a comment

    August 05, 2014
    Vandraketter (August 05, 2014, 05:00 UTC)

    Det har været en forrygende varm sommer. Hvad er bedre end lidt vandpjaskeri? Tilmed noget af den slags, man kan blive klogere på.

    Jeg har i længere tid været fan af Copenhagen Suborbitals. De har været gode til, at formidle raketvidenskab. raketvidenskab. Raketvidenskab er i folkemunde oftest ligmed noget ufatteligt svært. Det er måske ikke helt korrekt. Det er nærmere en vifte af videnskabsgenrer, som man skal kunne mestre på een gang. Det er her Copenhagen Suborbitals udemærker sig. De har fundet disse kompetancer i en række personer, de har formidlet deres viden, de har bygget og demonstreret det. De er måske 10 år fra deres mål, men de demonstrerer i høj grad fagligheden i ingeniørfaget.

    Raketvidenskab kan heldigvis også forstås tættere på jordhøjde og simplificeres så selv mindre børn kan være med. Vandraketter er stedet at begynde. Fysikken for en sådan trykfødet vandraket er den samme som for professionelle raketter.


    Jeg har bygget en såkaldt Gardena-affyringsrampe. Det er en haveslangekobling, der udløses via snoretræk. Opskriften fandt jeg i allerbedste Open Source stil på det australske site aircommandrockets.com. Mine sønner på 4 og 5 år nød at være en tur med i parken for at lege med vand. Det blev meget hurtigt for trangt i villahaven.

    vandraket højt til vejrs

    Den mest simple vandraket består af en enkelt plasflaske, hvor man på låget har pålimet en Gardena kobling. I praksis har jeg benyttet en billig kopi fra Harald Nyborg kaldet Adano i en rød model. Til affyringsplatformen valgte jeg dog en model i messing. frem for plast. Jeg bor tæt på både en Silvan, Jem&Fix og Harald Nyborg, så der burde ikke være langt til bilælige stumper. Jeg blev overrasket over hvor dyre sådanne simple plaststumper som haveslangekoblinger kan være. Det er ok at originalen Gardena er dyr, men det er langt ude at Silvans eget mærke Park er voldsomt dyre. Silvan putter for meget slamværktøj under deres egne brands til at kunne opretholde ideen om en dyr pris for deres kopivarer.

    Der er rigtig mange gode fif at finde på det australske site og mange andre steder på nettet når man søger efter water rockets.

    Den viste vandraket er en lidt udviddet version, hvor jeg har sat to plastflasker sammen. Jeg har her boret et 16mm hul i bunden af begge plastflasker, stukket en stump elektrikerrør på 16mm ind i begge bunde og limet og tætnet med klar silikone. Silikonen har jeg ladet tørre en dags tid før jeg har benyttet raketten.

    Som pumpe har jeg benyttet en billig cykelpumpe hos Harald Nyborg (FODPUMPE DOBBELTCYLINDRET Best.nr. 7158). Her kommer så det klassiske converterproblem. Hvordan bruger man en cykelpumpe til en haveslange? Min løsning var at klippe den medfølgende studs af.fjerne lidt af stofindpakningen af gummislangen og indføre den i mellemstykke til en haveslangekobling (Best nr. 6579). Jeg limede slangen fast i hele  koblingens længde med superlim.

    Jeg er nu nået til trin 2. Det oplagte at gå videre med er finner, der kan øge retningsstabiliteten. En mere aerodynamisk snude kunne også være et punkt. En faldskærm for at raketten kommer sikkert ned. En udløser for faldskærnen. Et webkamera ombord. Der er rigtigt meget at gå videre med. Jeg har allerede forsøgt en del og meget er fejlet. Nogle gange lærer man bare mere at et hurtigt testforsøg til få basører end hvad man kan læse eller regne sig frem til via lærerbøger og hjemmesider for andre entutiaster.

    Og det er netop det raketvidenskab går ud på. Den ene udfordring fører til den næste. Der er mange teknikker der skal mestres på een gang og de bliver hurtigt koblet tæt sammen. Udfordringen for mig er hvor længe jeg kan holde mig selv og mine børn inspireret og udfordret.

    Udfordringen for Copenhagen Suborbitals er hvor længe de kan holde resten af Danmark (og resten af verdenen) inspireret.

    Leave a comment

    August 01, 2014
    Poul-Henning Kamp a.k.a. phk
    Netværksfejl (August 01, 2014, 13:25 UTC)
    Poul-Henning Kamp Stort set hver gang et eller andet stort system går ned i timevis, viser det sig at være 'netværksfejl', men hvad ved vi egentlig om disse 'netværksfejl', og hvordan kan vi blive bedre til at designe systemer, der overlever dem ? ACM Queue har lige publiceret en rigtig interessant artikel fyldt ...

    Leave a comment

    July 16, 2014

    Baptiste Daroussin started the session with a status update on package building. All packages are now built with poudriere. The FreeBSD Foundation sponsored some large machines on which it takes around 16 hours to build a full tree. Each Wednesday at 01:00UTC the tree is snapshot and an incremental build is started for all supported released, the 2 stable branches (9 and 10) and quarterly branches for 9.x-RELEASE and 10.x-RELEASE. The catalogue is signed on a dedicated signing machine before upload. Packages can be downloaded from 4 mirrors (us-west, us-east, UK, and Russia) and feedback so far has been very positive.

    He went on to note that ports people need better coordination with src people on ABI breakage. We currently only support i386 and amd64, with future plans for ARM and a MIPS variant. Distfiles are not currently mirrored (since fixed), and while it has seen no progress, it’s still a good idea to build a pkg of the ports tree itself.

    pkg 1.3 will include a new solver, which will help 'pkg upgrade' understand that an old packages needs to be replaced with a newer one, with no more need for 'pkg set' and other chicanery. Cross building ports has been added to the ports tree, but is waiting for pkg-1.3. All the dangerous operations in pkg have now been sandboxed as well.

    EOL for pkg_tools has been set for September 1st. An errata notice has gone out that adds a default pkg.conf and keys to all supported branches, and nagging delays have been added to ports.

    Quarterly branches based on 3 month support cycle has been started on an evaluation basis. We’re still unsure about the manpower needed to maintain those. Every quarter a snapshot of the tree is created and only security fixed, build and runtime fixed, and upgrades to pkg are allowed to be committed to it. Using the MFH tag in a commit message will automatically send an approval request to portmgr and an mfh script on Tools/ makes it easy to do the merge.

    Experience so far has been good, some minor issues to the insufficient testing. MFHs should only contain the above mentioned fixes; cleanups and other improvements should be done in separate commits only to HEAD. A policy needs to be written and announced about this. Do we want to automatically merge VuXML commits, or just remove VuXML from the branch and only use the one in HEAD?

    A large number of new infrastructure changes have been introduces over the past few months, some of which require a huge migration of all ports. To speed these changes up, a new policy was set to allow some specific fixes to be committed without maintainer approval. Experience so far has been good, things actually are being fixed faster than before and not many maintainers have complained. There was agreement that the list of fixes allowed to be committed without explicit approval should be a specific whitelist published by portmgr, and not made too broad in scope.

    Erwin Lansing quickly measured the temperature of the room on changing the default protocol for fetching distils from MASTER_SITE_BACKUP from ftp to http. Agreement all around and erwin committed the change.

    Ben Kaduk gave an introduction and update on MIT’s Athena Environment with some food for thought. While currently not FreeBSD based, he would like to see it become so. Based on debian/ubuntu and rolled out on hundreds of machines, it now has it’s software split into about 150 different packages and metapackages.

    Dag-Erling Smørgrav discussed changes to how dependencies are handled, especially splitting dependencies that are needed at install time (or staging time) and those needed at run time. This may break several things, but pkg-1.3 will come with better dependency tracking solving part of the problem.

    Ed Maste presented the idea of “package transparency”, loosely based on Google’s Certificate Transparency. By logging certificate issuance to a log server, which can be publicly checked, domain owners can search for certificates issued for their domains, and notice when a certificate is issued without their authority. Can this model be extended to packages? Mostly useful for individually signed packages, while we currently only sign the catalogue. Can we do this with the current infrastructure?

    Stacy Son gave an update on Qemu user mode, which is now working with Qemu 2.0.0. Both static and dynamic binaries are supported, though only a handful of system call are supported.

    Baptiste introduced the idea of having pre-/post-install scripts be a library of services, like Casper, for common actions. This reduces the ability of maintainers to perform arbitrary actions and can be sandboxed easily. This would be a huge security improvement and could also enhance performance.

    Cross building is coming along quite well and most of the tree should be able to be build by a simple 'make package'. Major blockers include perl and python.

    Bryan Drewery talked about a design for a PortsCI system. The idea is that committer easily can schedule a build, be it an exp-run, reference, QAT, or other, either via a web interface or something similar to a pull request, which can fire off a build.

    Steve Wills talked about using Jenkins for ports. The current system polls SVN for commits and batches several changes together for a build. It uses 8 bhyve VMs instances, but is slow. Sean Bruno commented that there are several package building clusters right now, can they be unified? Also how much hardware would be needed to speed up Jenkins? We could duse Jenkins as a fronted for the system Bryan just talked about. Also, it should be able to integrate with phabricator.

    Erwin opened up the floor to talk about freebsd-version(1) once more. It was introduced as a mechanism to find out the version of user land currently running as uname -r only represents the kernel version, and would thus miss updates of the base system that do no touch the kernel. Unfortunately, freebsd-version(1) cannot really be used like this in all cases, it may work for freebsd-update, but not in general. No real solution was found this time either.

    The session ended with a discussion about packaging the base system. It’s a target for FreeBSD 11, but lots of questions are still to be answered. What granularity to use? What should be packages into how many packages? How to handle options? Where do we put the metadata for this? How do upgrades work? How to replace shared libraries in multiuser mode? This part also included the quote of the day: “Our buildsystem is not a paragon of configurability, but a bunch of hacks that annoyed people the most.”

    Thanks to all who participated in the working group, and thanks again to DK Hostmaster for sponsoring my trip to BSDCan this year, and see you at the Ports and Packages WG meet up at EuroBSDCon in Sofia in September.

    Leave a comment

    The DNS Working Group at the FreeBSD Developer Summit at BSDCan this year was off to a good start by noticing that DNSSEC validation could not work on the University of Ottawa’s wireless network. The university’s resolvers added additional records to the root zone, thus failing validation at the root. This led to some discussion on how to provide a user-friendly way to explain this in an understandable way to the user and giver the user a choice of turning off validation or find another network. This certainly is going to be a major problem when turning on validation by default as broken resolvers are very common at hotels, coffee shops, etc. etc.

    On a more positive note, all the FreeBSD projects zones are DNSSEC signed and all project-owned servers have SSHFP records in the zone. Dog food was eaten.

    Dag-Erling Smørgrav started off by giving an overview of the current state of affairs. ldns and unbound are imported into base in HEAD and 10.x. unbound is meant to act as a local resolver only and as it is not linked to libevent, it will not scale to anything else. For a network-wide resolver or any other configuration, it is recommended to install unbound from ports. DES further went into some of the implementation details on how the base unbound is installed to make sure it does not conflict with an unbound installed from ports.

    DES explained some issues he encountered with local and RFC1918 zones which are filtered by default by unbound. Others reported no issues with the right configuration options, so more investigation is needed.

    Some people reported having difficulty getting patches accepted upstream by NLNetLabs, which gave some cause for concern as we clearly want a good and active working relationship with our DNS vendor. Others reported no problem working with NLNetLabs, quite the opposite, they are very interested to see the work going on in operation systems, so we’ll just need to build upon that relationship and make sure to invite them to the next WG meeting. Patches that are currently being worked on, DES has some code cleanups, Björn a DNS64 feature, should be submitted through the “normal” submission process and review with NLNetLabs and we’ll see how that goes.

    Erwin Lansing started the brainstorm session on future work. Some command line tools would be nice to have; drill does most things one wants, but people are too used to writing dig and dig has many more options; Peter Wemm would like to see contrib scripts line ldns-dane, which are just really easy to use; the control socket should be a unix socket, there’s a patch floating around and should be submitted upstream.

    The “Starbucks” problem came up again, with a proposal to turn on val-permissive-mode by default. Another solution may be by looking at how unbound-trigger does its magic.

    After a coffee break, Peter Losher, ISC, went over some of the recent changes at ISC. BIND10 development has been handed over to a new project and ISC will concentrate on BIND9 and a stand-alone project for the DHCP component. BIND 9.10 was recently released and plans are in place for 9.11. ISC is open to suggestions and feature requests.

    Peter brought up the topic of clientID for which a IETF draft (draft-edns0-client-subnet) is available. This would help client find the nearest CDN node, etc. ISC wants this to be an opt-out in operating systems as it will peel off a layer of anonymisation, and should be controllable by the user.

    Next up was Michael Bentkofsky, Verisign, who, while not involved in the project himself, gave an introduction into the getDNS API, which is a replacement for getaddrinfo and allows the stub resolver to get validation information down at the client level. It’s available in ports. The discussion went into more of a brainstorm on how applications should get DNS and DNSSEC information and who gets to make decisions about its security. There should be a clear separation between policy and mechanism, where application programmers should not have to worry about this; it should be a system policy. There should be a higher level API where an application basically can ask the operating system for a “connection” and the operation system takes care of everything behind the scenes, DNS, DNSSEC, SSL, DANE, etc. and just return a socket, with some information on how the connection was established and which security mechanisms were used. In FreeBSD, it would make sense to let the Casper daemon hand out the different sub-tasks to ensure all lookups, cryptography, etc. are properly compartmentalised. One potential problem with passing on additional information is that all DNS lookups currently go through nsswitch, which would need to grow knowledge about that data as well. Are people still using other mechanisms for hostname lookups besides the hosts file and DNS? We can probably just remove nsswitch for the hostname lookups.

    The session ended with some aims for the 11.0 release. We’ll need to have a wider discussion about the aforementioned removal of nsswitch out of the hostname lookups. We’ll also need a better understanding of what API capabilities applications may need. Can Casper provide all these? Can it run unbound behind the scenes to do all the DNS “stuff” for it? Can we capsicumize unbound and will that be accepted upstream? Enough food for thought and even more for writing code.

    Thanks again to DK Hostmaster for sponsoring my trip to BSDCan this year, and see you at the DNS WG meet up at EuroBSDCon in Sofia in September.

    Leave a comment

    Elasticsearch graphs (July 16, 2014, 07:35 UTC)

    After having worked with Elasticsearch and thrown quite a lot of data at it (we add about 100 million documents a day), I have built a very nice set of graphs, that helps me visualize problems and activity in the cluster, and figured I'd share them to hopefully give some inspiration :)

    p.s. the jvm_heap_usage graphs - the two lines which are very jumpy, are the ones I switched to using G1 Garbage Collector, which does seem to be of help when you're running close to your heap limit :) 

    p.s. view image alone, to see it in full size.

    read more

    Leave a comment

    July 15, 2014
    Datafællesskabet data.coop (July 15, 2014, 11:52 UTC)

    For tre uger side, tirsdag d. 24. juni, afholdtes stiftende generalforsamling i data.coop.

    Foreningens formål er ifølge vedtægterne:


    Foreningen data.coop ønsker at stille digital infrastruktur til rådighed for sine medlemmer, på en måde hvor foreningens kerneprincipper — privatlivsbeskyttelse, kryptering, decentralisering og zero-knowledge for foreningen som tjenesteudbyder — er i fokus. Ydermere vil foreningen advokere for sine kerneprincipper, hjælpe folk til at at agere på nettet på forsvarlig vis, samt samarbejde med andre datafællesskaber/hjælpe andre i gang med lign. foreninger.

    Motivationen for stiftelse af foreningens formuleres nok bedst af Mikkel her: https://www.detfalskested.dk/2014/06/18/indkaldelse-til-stiftende-generalforsamling-for-datafaellesskab/

    Efter en længere process på den stiftende generalforsamling med småændringer til et sæt standardvedtægter blev disse endelig godkendt, og der blev valgt en bestyrelse (og yours truly fik tilranet sig en plads).



    …hvad så nu?

    Lige nu er der vist gået sommer(ferie) i den.

    Men på et tidspunkt skal der selvfølgelig gang i det praktiske datafællesskab. Første trin er vist en fornuftig e-mail løsning. Hvad der derefter komme af data-hosting, Dropbox alternativ eller lignende må tiden vise.
    Hvilke konkret tekniske løsninger der skal benyttes, hvor data skal hostes osv. vil jeg slet ikke forholde mig til, men det er min oplevelse at der er blevet tænkt en del over dette af folk med mere teknisk indsigt end jeg.

    Bestyrelsen skal nok også afholde et møde, forholde sig til eventuel økonomi og lignende.

    Der bør nok også oprettes en hjemmeside, med mere udførlig kontakt-info, mail-lister osv. Det arbejde er vist også så småt sat i gang.

    Hvis du har lyst til at være med er det letteste nok at holde øje med hjemmesiden data.coop, hvor der sikkert dukker yderligere info op i den nærmeste fremtid. Alternativt kan du forsøge at at kontakte (medlemmer af) bestyrelsen.

    Image by: Bob Mical

    July 09, 2014
    Sune Vuorela a.k.a. pusling
    CMake and library properties (July 09, 2014, 06:30 UTC)
    Sune Vuorela

    When writing libraries with CMake, you need to set a couple of properties, especially the VERSION and SOVERSION properties. For library libbar, it could look like:

    set_property(TARGET bar PROPERTY VERSION “0.0.0″)
    set_property(TARGET bar PROPERTY SOVERSION 0 )

    This will give you a libbar.so => libbar.so.0 => libbar.so.0.0.0 symlink chain with a SONAME of libbar.so.0 encoded into the library.

    The SOVERSION target property controls the number in the middle part of the symlink chain as well as the numeric part of the SONAME encoded into the library. The VERSION target property controls the last part of the last element of the symlink chain

    This also means that the first part of VERSION should match what you put in SOVERSION to avoid surprises for others and for the future you.

    Both these properties control “Technical parts” and should be looked at from a technical perspective. They should not be used for the ‘version of the software’, but purely for the technical versioning of the library.

    In the kdeexamples git repository, it is handled like this:


    And a bit later:

    set_target_properties(bar PROPERTIES VERSION ${BAR_VERSION}

    which is a fine way to ensure that things actually matches.

    Oh. And these components is not something that should be inherited from other external projects.

    So people, please be careful to use these correct.

    Leave a comment

    July 03, 2014
    Poul-Henning Kamp a.k.a. phk
    Og jeg gentager: (July 03, 2014, 10:27 UTC)
    Poul-Henning Kamp Nej, jeg er faktisk så træt af at gentage mig selv at jeg ikke gør det. Kan I ikke bare finde nogen af mine tidligere brok om IT havarikommission og om hvorfor mainframe-miljøer er en sikkerhedstrussel i arkivet ? Og ja, jeg synes det er i særklasse ironisk at det er netop de borgere der har gj...

    Leave a comment

    July 02, 2014
    Peter Toft a.k.a. pto
    Peter Toft Jeg modtager - sikkert ligesom jer - et hav af emails, der prøver at franarre mig penge. I dag fik jeg en, jeg var "tættere" på at tro på. Personen - vi kan kalde ham Kim - er en bekendt, som sagtens kunne tænkes at være den rigtige afsender. I dette tilfælde kontaktede jeg personen, som kunne b...

    Leave a comment

    The calculations: 10Gbit/s wirespeed (July 02, 2014, 10:18 UTC)
    In this blogpost, I'll try to make you understand the engineering challenge behind processing 10Gbit/s wirespeed, at the smallest Ethernet packet size.

    The peak packet rate is 14.88 Mpps (million packets per sec) uni-directional on 10Gbit/s with the smallest frame size.

    Details: What is the smalles Ethernet frame
    Ethernet frame overhead:

    Thus, the minimim size Ethernet frame is: 84 bytes (20 + 64)

    Max 1500 bytes MTU Ethernetframe size is: 1538 bytes (calc: (12+8) + (14) + 1500 + (4) = 1538 bytes)

    Packet rate calculations

    Peak packet rate calculated as:  (10*10^9) bits/sec / (84 bytes * 8) = 14,880,952 pps
    1500 MTU packet rate calculated as: (10*10^9) bits/sec / (1538 bytes * 8) = 812,744 pps

    Time budget
    This is the important part to wrap-your-head around.

    With 14.88 Mpps the time budget for processing a single packet is:

    • 67.2 ns (nanosecond) (calc as: 1/14880952*10^9 ns)

    This corrospond to approx: 201 CPU cycles on a 3GHz CPU (assuming only one instruction per cycle, disregarding superscalar/pipelined CPUs). Only having 201 clock-cycles processing time per packet is very little.

    Relate these numbers to something
    This 67.2ns number is hard to use for anything, if we cannot relate this to some other time measurements.

    A single cache-miss takes: 32 ns (measured on a E5-2650 CPU). Thus, with just two cache-misses (2x32=64ns), almost the total 67.2 ns budget is gone. The Linux skb (sk_buff) is 4 cache-lines (on 64-bit), and the kernel e.g. insists on writing zeros to these cache-lines, during allocation of an skb.

    We might not "suffer" a full cache-miss, sometimes the memory is available in L2 or L3 cache.  Thus, it is useful to know these time measurements.  Measured on my E5-2630 CPU (with lmbench command "lat_mem_rd 1024 128"), L2 access costs 4.3ns, and L3 access costs 7.9ns.

    The "LOCK" operation
    Assembler instructions can be prefixed with a "LOCK" operation, which means that they perform an atomic operation. This is uses every time e.g. a spinlock is locked or unlocked, cmpxchg and atomic_inc (some operations are even implicitly LOCK prefixed, like xchg).

    I've measured the cost of this atomic "LOCK" operation to be 8.25ns on my CPU (with this program). Even for the completely optimal situation of a spinlock only being touch by one CPU, we have two LOCK calls which costs 16.5ns.

    System call overhead
    A FreeBSD case study of sendto(), in Luigi Rizzo netmap paper, shows that the cost of only the system call is 96ns, which is above the 67.2 ns budget.  The total overhead of sendto() were 950 ns.  These 950ns corrospond to 1,052,631 pps (calc as 1/(950/10^9)).
    On Linux I measured the system call getuid(2), to take 87.77 ns and 201 CPU-cycles (TSC measurement) (the CPU efficiency were 1.42 insns per cycle, measured with perf stat). Thus, the syscall itself eats up the entire budget.

    • Update: Most of the syscall overhead comes from kernel option CONFIG_AUDITSYSCALL, without it, the syscall overhead drops to 41.85 ns.

    How to overcome this syscall problem?  We can amortize the cost, by sending several packets in a single syscall.  It is not very well known, but we actually already have a syscall to send several packets with a single syscall, called "sendmmsg(2)". Notice the extra "m" (and the corresponding receive version "recvmmsg(2)"). Not many examples exists on the Internet for using these syscalls. Thus, I've provided some example code here for sendmmsg and recvmmsg.

    RAW socket speeds
    Daniel Borkmann and I recently optimized AF_PACKET, to scale to several CPUs (trafgen, kernel qdisc bypass and trafgen use qdisc bypass). But let us look at the performance numbers for only a single CPU:

    • Qdisc path = 1,226,776 pps => 815 ns per packet (calc: 1/pps*10^9)
    • Qdisc bypass = 1,382,075 pps => 723 ns per packet (calc: 1/pps*10^9)

    This is also interesting, because this show us the cost of the qdisc code path, which costs 92 ns.  In this 10Gbit/s context it is fairly large, e.g. corresponding to almost 3 cache-line misses (92/32=2.9).

    Poul-Henning Kamp a.k.a. phk
    Gettys principper (July 02, 2014, 08:18 UTC)
    Poul-Henning Kamp Jeg sider og prøver at stoppe noget sund fornuft ind i HTTP/2.0 standardiseringsprocessen. Det er hårdt arbejde som ville være nemmere hvis flere mennesker kendte og respekterede "Gettys Regler" For rigtig mange år siden formulerede Jim Gettys nogle grundprincipper for X11 udviklingen, som desv...

    Leave a comment

    June 29, 2014
    Martin Schlander a.k.a. cb400f
    Jolla and KDE Connect (June 29, 2014, 16:08 UTC)
    Martin Schlander

    KDE Connect

    KDE Connect is a piece of software that integrates your KDE desktop with Android devices. It enables you to share the clipboard, share files, use your Android device as a mousepad or remote control for MPRIS enabled media players on your desktop, have a battery indicator for your Android device on your desktop and more. Even more features are planned. All this is done over wifi.


    Jolla is of course the coolest smartphone on the market, it runs SailfishOS, but it comes with an Android runtime (Alien Dalvik) which lets you run most Android apps perfectly fine on the Jolla.

    KDE Connect on the Jolla

    So I had to see if KDE Connect would work with the Jolla, and at least some of the main features work perfectly. I can now use my Jolla as a wireless mousepad for my KDE desktops, and I can use my Jolla as a remote control for e.g. Amarok. I can also work with the filesystem in the Dolphin file manager, but only the Android runtime folders of the Jolla filesystem are exposed to KDE this way.

    Quite a few of the features don’t seem to work – notifications, battery indicator, sending files via the Dolphin context menu (right click) and clipboard sharing.

    Media Player Remote Control


    How to set it up

    1) Install KDE Connect on your desktop (on openSUSE install the package ‘kdeconnect-kde’ from the KDE:Extra repository. Also install ‘sshfs’ if you want to be able to mount the Android folders on the Jolla in Dolphin.

    2) Install KDE Connect on your Jolla (personally I installed the binaries from the F-Droid app store, but binaries are also available in Google Play and 1MobileMarket).

    3) Connect your Jolla to the wifi of the same network as your desktop computer and make sure you don’t have a firewall running (or allow traffic for the range of ports 1714-1764 for both TCP and UDP).

    4) Launch the KDE Connect app on the Jolla and go to KDE ‘systemsettings’ -> KDE Connect and pair your phone with the desktop.

    June 27, 2014
    Poul-Henning Kamp a.k.a. phk
    Historiske IT success/katastrofer (June 27, 2014, 07:07 UTC)
    Poul-Henning Kamp Der existerer en konference for historisk IT i de nordiske lande, "HiNC" og den når til Danmark d. 13-15 august. Det er ret fantastisk hvor vidt omkring programmet kommer, fra megasuccessen CPR over "APL i de nordiske lande" til gigantfiaskoen EPJ. Jeg er godt klar over at det ikke er alle der ...

    Leave a comment

    June 25, 2014
    Peter Toft a.k.a. pto
    Peter Toft Jeg holder meget af at programmere i Python. Det er klart det bedste programmeringssprog, jeg har arbejdet med. Det er to ting jeg jævnligt har brug for - at finde ud af hvor i min kode, jeg bruger mest CPU-kraft hhv. mest hukommelse. Til C/C++ kode har jeg meget godt styr på det men til Python k...

    Leave a comment

    Poul-Henning Kamp a.k.a. phk
    En fyr med en god tidsmaskine... (June 25, 2014, 08:54 UTC)
    Poul-Henning Kamp ...afslører hvordan Keynote foredraget lyder til Perl konferencen 2034. Fremtiden er ikke hvad vi blev lovet. Det relevante spørgsmål er ikke "tidsmaskine ?" eller "Perl konferencen 2034 ?!" men "Er det den verden vi vil leve i?" Charles Stross styrke som "near-term" science fiction forfatter...

    Leave a comment

    June 13, 2014
    Omvendt betalingspligt (June 13, 2014, 06:28 UTC)

    Vores regering arbejder hård for at lette de administrative byrder, påstår de.

    Men virkeligheden er en ganske anden.

    Momslovens regler om omvendt betalingspligt kan kun opfattes som ren chikane.

    Nedenstående er sakset fra http://www.bakertilly.dk/sidste-nyt/vejledning-til-omvendt-betalingspligt/

    Døm selv…

    June 11, 2014
    Kenneth Geisshirt a.k.a. kneth
    Emacsforum 2011 (June 11, 2014, 19:06 UTC)
    Kenneth Geisshirt Emacsforum 2011
    Peter Toft and I are in the process of preparing Emacsforum 2011 with some help by Troels Henriksen (at DIKU) and Keld Simonsen (from KLID). The program is almost ready for publication, so I will not say too much - but there will be something for scientists and developers. Even our Evil Twin will be represented.

    The mini-conference takes place 12th November 2011 at DIKU. The is no conference fee - and there will be no benifits.

    If you are using Emacs (and even XEmacs) and live in the Copenhagen area, Emacsforum is a good place to meet fellow users.
    Poul-Henning Kamp a.k.a. phk
    Dronningens Trojanske Cybergarde (June 11, 2014, 08:35 UTC)
    Poul-Henning Kamp Alt tyder på at Folketinget vedtager den Forsvarets nye ceremonielle cybergarde idag. Lovforslag L.192 indeholder i bund og grund hjemmel til at stille en soldat på parade ved alle offentlige IT-systemers firewalls hvor han kan stå og se om nogen turister prøver at komme forbi. Den absolut mest...

    Leave a comment

    June 08, 2014
    Flere i arbejde. (June 08, 2014, 18:13 UTC)

    Vi har stadig mange uden arbejde. Det kan vi gøre noget ved.

    1. Fjern momsen på ydelser. Det mistede provenu vil vil komme igen i form af besparelser på overførselsindkomst og øgede skatteindtægter. Flere i arbejde og bedre betalingsbalance.

    2. Erstat ejendomsværdiskatten af fast ejendom med skat af fortjenesten ved ejendomshandel – fratrukket dokumenterede udgifter til forbedringer og vedligehold. Så bliver værditilvækst skabt af sort arbejde beskattet = mere hvidt og mindre sort.

    3. Hæv reparationsgrænsen for totalskade af motorkøretøjer fra de nuværende 75% til 100% af værdien. Det vil give arbejde til rigtigt mange pladesmede og mindre import af nye køretøjer. Flere i arbejde, bedre betalingsbalance og mindre miljøbelastning.

    4. Erstat licensbaseret software i den offentlige sektor med fri software og brug den årlige besparelse på mere end 3 mia. til at forbedre denne software. Flere i arbejde, højere vidensniveau, bedre betalingsbalance og højere national sikkerhed.

    Find selv på flere – det er ikke så svært.

    June 05, 2014
    Poul-Henning Kamp a.k.a. phk
    Behovet for fundering... (June 05, 2014, 08:14 UTC)
    Poul-Henning Kamp Idag er Grundlovsdag og politikere kværner løs med floskler osv. Grundloven hedder sådan fordi de er fundamentet under vores civilization, i bund og grund er den det eneste der forhindrer mig i at myrde folk jeg ikke er enig med, resten af lovene er bare detaljelovgivning der skal give Grundlove...

    Leave a comment

    June 04, 2014
    Pktgen for network overload testing (June 04, 2014, 17:38 UTC)
    Want to get maximum performance out of the kernel level packet generator (pktgen)?
    Then read this blogpost:

    • Simple tuning will increase performance from 4Mpps to 5.5Mpps (per CPU)

    You might see pktgen as a fast packet generator, which it is, but I (as a kernel developer) also see it as network stack testing tool of the TX code path.

    Pktgen have a parameter "clone_skb", which specifies how many time to send the same packet, before freeing and allocting a new packet for transmission.  This affects performance significantly, as it can remove a lot of memory allocation and access overhead.

    I have two distinctly different use-cases for stack testing:

    1. clone_skb=1      tests the stack alloc/free overhead (related to the SKB)
    2. clone_skb=100000 tests the NIC driver layer
    Lets focus on case 2, driver layer.

    Tuning NIC driver layer for max performance:
    The default NIC setting are not tuned for pktgen's artificial overload type of benchmarking, as this could hurt the normal use-case.

    Specifically increasing the TX ring buffer in the NIC:
     # ethtool -G ethX tx 1024

    A larger TX ring can improve pktgen's performance, while it can hurt in the general case, 1) because the TX ring buffer might get larger than the CPUs L1/L2 cache, 2) because it allow more queueing in the NIC HW layer (which is bad for bufferbloat).

    One should be careful to conclude, that packets/descriptors in the HW TX ring cause delay.  Drivers usually delay cleaning up the ring-buffers (for various performance reasons), thus packets stalling the TX ring, might just be waiting for cleanup.

    This "slow" cleanup issues is specifically the case, for the driver ixgbe (Intel 82599 chip).  This driver (ixgbe) combine TX+RX ring cleanups, and the cleanup interval is affected by the ethtool --coalesce setting of parameter "rx-usecs".

    For ixgbe use e.g "30" resulting in approx 33K interrupts/sec (1/30*10^6):
     # ethtool -C ethX rx-usecs 30

    Performance data:
    Packet Per Sec (pps) performance tests using a single pktgen CPU thread, CPU E5-2630, 10Gbit/s driver ixgbe. (using net-next development kernel v3.15-rc1-2680-g6623b41)

    Adjusting the "ethtool -C ethX rx-usecs" value affect how often we cleanup the ring.  Keeping the default TX ring size at 512, and adjusting "rx-usecs":
    • 3,935,002 pps - rx-usecs:  1 (irqs:  9346)
    • 5,132,350 pps - rx-usecs: 10 (irqs: 99157)
    • 5,375,111 pps - rx-usecs: 20 (irqs: 50154)
    • 5,454,050 pps - rx-usecs: 30 (irqs: 33872)
    • 5,496,320 pps - rx-usecs: 40 (irqs: 26197)
    • 5,502,510 pps - rx-usecs: 50 (irqs: 21527)
    Performance when adjusting the TX ring buffer size. Keeping "rx-usecs==1" (default) while adjusting TX ring size (ethtool -G):
    • 3,935,002 pps - tx-size:  512
    • 5,354,401 pps - tx-size:  768
    • 5,356,847 pps - tx-size: 1024
    • 5,327,595 pps - tx-size: 1536
    • 5,356,779 pps - tx-size: 2048
    • 5,353,438 pps - tx-size: 4096
    The performance of adjusting cleanup interval (rx-usecs), seems to win over simply increasing the TX ring buffer size. This also proves the theory of TX queue is filled with old packets/descriptors that needs cleaning.
    (Edit: updated numbers to be clean upstream, previously included some patches)

    Tools: Want easy to use script for pktgen look here
    More details on pktgen advanced topics by Daniel Turull.
    June 03, 2014
    Sune Vuorela a.k.a. pusling
    Bringing KDE forward (June 03, 2014, 20:31 UTC)
    Sune Vuorela

    The almost-yearly large KDE-sprint in Randa, Switzerland is doing a fundraiser to get this year’s event running. See http://www.kde.org/fundraisers/randameetings2014/

    This year, it is besides the recurring multimedia topics, a lot about improving the new KDE Frameworks, the related documentation and the development experience with IDE’s and such.

    It is also a good way to come full circle, since it was back in 2011 when I was at the Randa Meetings that the KDE Frameworks initiative was started.

    So once again: http://www.kde.org/fundraisers/randameetings2014/

    Leave a comment

    Poul-Henning Kamp a.k.a. phk
    SCOTUS om patenter, fortsat... (June 03, 2014, 07:22 UTC)
    Poul-Henning Kamp I hele denne sæson har USAs Højesteret haft en række patentsager på programmet og der faldt dom i to af dem igår. Limelight Networks, Inc. v. Akamai Technologies, Inc. Nautilus, Inc. v. Biosig Instruments, Inc og i begge sager, som i de forrige, får den særlige Patent-appel-ret ("Federal Circu...

    Leave a comment

    June 02, 2014
    Vores hjem kan købes. (June 02, 2014, 04:46 UTC)

    Nu hvor 2 af vores børn er flyttet hjemmefra og den sidste rejser på udveksling har vi besluttet at give plads til en ny familie.

    Vi taler om en dejlig sund villa på Smedievej 66 i Hillerød på 197 m2. Blandt de mange ting som gør af vi nydt at bo her, kan nævnes:

    Børnevenligt område med stisystemer, tæt på skole, daginstitution og offentlig transport.

    Dejlig lys og rummelig stue hvor vi bl.a. har holdt konfirmationer med ikke mindre end 45 til bords.

    3 gode værelser i stueplan og 3 på 1. sal, det ene med altan. Mulighed for 7 gode værelser ved opdeling af det største.

    Lækkert marmorbadeværelse med brus i stueplan og baderum med kar på 1. sal.

    Velfungerende og godt indrettet Modulia kvalitetskøkken med god aflægger plads og nyere hvidevarer.

    2 toiletter i stueplan.

    Isoleret loft med god plads til de ting I ikke bruger til daglig.

    Dejlig overdækket terrasse med mørke fliser som suger varmen om dagen og holder lunt om aftenen, så I kan nyde udelivet  fra april til oktober.

    Dejlig stor indkørsel med herregårdssten.

    Kæmpe carport med plads til både campingvogn og 2 biler.

    Isoleret og opvarmet hobbyværksted på 9 m2.

    Dejlig blomstrende ugeneret have med mange gode kroge, frugttræer og fuglesang.

    Huset er opvarmet med billig fjernvarme. Der er også en brændeovn til en lun hyggeaften.

    Alt er velholdt og og i god stand. Lige klar til indflytning!

    I kan se nogle billeder ved at klikke her

    Det kan blive jeres for kr. 2.875.000,-

    Se mere på http://www.selvsalg.dk/bolig/11958/smedievej_66-3400-hilleroed

    Nysgerrig? Så ring på 20669860.



    May 30, 2014
    Peter Toft a.k.a. pto
    Peter Toft Jeg så en interessant lille historie i går. En dansker (Jon Clausen) modtager en phishing-mail svarende til denne. Det skal man naturligvis ALDRIG gøre. Det er fup og svindel! Da Jon fik mailen tænkte han sig lidt mere om. Han ville sende dem videre til NemID sammen med de tilhørende log ent...

    Leave a comment