Turing and the Enigma, again, and What about standard file formats?

[peter.s]

I

Since the public viewing of the Snowden affair is 1 year old, there are some articles in the press, on Snowden, but also on Turing and encryption/decryption, and I stumbled on this one (in German): http://blog.zeit.de/...a-zweiter-weltkrieg/

I never understood the Enigma, especially since every "explanation" about it you can find in the web, either is written by experts for experts, or by non-experts who don't understand the Enigma themselves, and the link above falls into the second category, but some comments there rise some interesting points.

It seems Turing began his decryption work on the Enigma after one functional Enigma machine fell into British hands.

It seems the "code for the day" was communicated using the previous code.

It seems some primary code needed for the real encryption, then to be made by the machine, with the help of the "code for the day", was a stable arrangement of the abc chars, and the British tried myriads of possible sequences for this, e.g. qwerty, and so on, but the Germans, "incredibly", just used "abcde", and it seems the mathematician Marian Rejewski did find out this, not Turing, and the commenter in the above link who brings this info into the discussion, muses that Rejewski had worked in Göttingen, Germany, beforehand, so had some first-hand info on German psychology / way of thinking, which enabled him to take into consideration the Germans might do it in the utmost basic, primitive way, a possibility excluded by Brits just admiring the machine but without intimate knowledge of German "national character" - I very much like this observation.

(EDIT: And of course, this over-emphasizing/relying upon the over-obvious resp. the "really-too-easy" reminds us of that E.A. Poe short story...)

It seems the breakthrough was then made by Turing's reflection that by the way the machine obviously worked, on a physical level - direct current was sent thru the rolls in one direction, then in the other direction - no character (a...z, etc.) could be replaced by itself, this drastically reducing the machine's encryption possibilities / possible permutations; in fact, the cited article is primarily about this phenomenon of "Selbst-Bewichtelung", no English translation found, just this transcription, "Players who receive their own gift in 2/3 of all secret Santa games."

Some commenter over there claims the biggest U.S. employer for mathematicians is the NSA - very funny and very convincing, even while no proof is given.

II

I, just some days ago, had mused about specific file formats countering effective encryption. Let's say you use MS Word files, or some other file formats where quite lengthy passages are, more or less identical, but highly standardized at the very least, for every one of your encrypted files, AND the decryptor knows (or can safely assume, from the presence of these applications on your system, or simply by the ubiquity of some applications, like MS Word, and its rather few "replacements", ditto for spreadsheets, etc.) which (few) applications you will have used to produce the encrypted data:

Then, this might drastically reduce the theoretical power of your specific encryption, since the decryptor (assuming even he doesn't have a way (which might exist, without us being aware of such possibilities) to determine where one of your file ends, and the next one begins, which would further cut the possible permutations into a mere fraction of their theoretical potential-by-strong-password) would try to decrypt those "standard passages" first, and even allowing for your individual data within these "standard passages", intimately knowing the "format" of the latter, incl. possible lengths of different such individual data in-between, and once these "file headers" are decrypted, your key will be known.

This would mean that usage of any application not producing just naked ansi files, but putting "processing data", "meta data" into the file, too, should be prohibited if you really want your data safe (= necessary but not sufficient condition)...

[mouser]

Peter, there are some wonderful books on encryption that I'm sure you would enjoy, that discuss both the historical and scientific aspects of the field.

Here's one I read recently that I can recommend:

The Code Book: The Science of Secrecy from Ancient Egypt to Quantum Cryptography Paperback, by Simon Singh

Simon Singh offers the first sweeping history of encryption, tracing its evolution and revealing the dramatic effects codes have had on wars, nations, and individual lives. From Mary, Queen of Scots, trapped by her own code, to the Navajo Code Talkers who helped the Allies win World War II, to the incredible (and incredibly simple) logisitical breakthrough that made Internet commerce secure, The Code Book tells the story of the most powerful intellectual weapon ever known: secrecy.  Throughout the text are clear technical and mathematical explanations, and portraits of the remarkable personalities who wrote and broke the world's most difficult codes. Accessible, compelling, and remarkably far-reaching, this book will forever alter your view of history and what drives it.

Of course Turing and the Enigma are discussed in the book.

[peter.s]

mouser, my point was that from the web alone, it's quite difficult to get some valid, comprehensive AND comprehensible, whilst concise info on these matters, short of reading books, and not being an expert in that field to begin with.

But this book has got unanimous rave reviews on all Amazon sites, so I thank you very kindly for your recommandation to what seems to be the finest intro into that matter available.

Thus, I just tried to get it from the otherwise incredibly good German inter-university library book lending system, and you know what? There are 3 hits for all of Germany, and with no lending, and of course, dozens of hits, with lending, for the German translation (and even one in French).

Of course, I could buy (or could I, in Europe, from amazon.com?) the (cheap) e-book, but I'm not into ebooks yet, even less so into reading 400-pages books on-screen; and of course, I could buy the English version, in a Continental bookstore, for about 50€ (= 70$) instead of 12$, at least that's been my experience from the old days.

Thus, there's always amazon.co.uk, where it's a mere 12$ plus postage, so I need to wait for other English books I'll need to complement my parcel.

Thus, my implicated "bad availability of readily available whilst not-too-basic info" argument wasn't entirely wrong, even incl. books, except if you accept translations which lately I don't do as easily anymore, since recurring reconnaissance of (too) literal translations harm my reading speed, real good translations being rare.

After all this OT whining, I'm sure your book recommendation is spot-on, and thank you so much again... and I'll bother fellow readers here again with the subject after having read the book. ;-)

[tomos]

^ not sure where you are Peter, but on amazon.de it's listed for ~ 10 euros new including delivery, (international or German depending on seller).
http://www.amazon.de...F8&condition=new

Actually I'm not 100% sure that's the same book - it's named "The Code Book: The Secret History of Codes and Code-breaking [Englisch] [Taschenbuch]"
got to it from this page for the hardback book (which lists the title as above)
http://www.amazon.de...Quantum+Cryptography

[peter.s]

OMG!

Thank you so much, tomos, I, too, often overlook the obvious: The English softcover is sold by amazon.de, too (and at an entirely acceptable price that is).

Often, you have to pay close attention with English titles on amazon.de, since (which is not the fault of amazon, but of the respective distributors) they sell numerous cinematographic film dvd's, with the original English title... and with JUST the German soundtrack, not even English subtitles then, which is outrageous: English titles sell the film, instead of giving it some idiotic German title, and then you will get some idiotic German synch only! (Some distributors do this on a systematic level, and in France, it's similar, and worse, since French synchs are systematically, abysmally laughable.) And then, with novels and such, this is recurrent, too: They often take the original English title for their translated books... (Not that I'd have time to read novels anymore, though...)

But in this instance, I was over-paranoid, and overlooked the native English offering, and it should be easy to find some other title to get to 20€ and have free delivery included, thank you so much again!


Radically OT:

Another totally OT hint to people living near some (any) German frontier: Anybody can get, from the German postal office, some "Gold Card", at the condition of having them your real identity, by passport, and you need a German (!) pre-paid mobile phone card (which is available to anybody, too), and then parcels from Germany, e.g. from amazon.de, are not to "John Smith, your home address abroad" anymore (which could cost a fortune, in some instances, or would not even be available), but "John Smith 12345678901234567890 (some long number), Packstation 123 (some short number, for an automatic delivery station), 12345 (postcode), town (name of some frontier-near town, on the German side of the frontier of course), and then, your problem will consist in finding suppliers who (like amazon and about 3/4 overall, but many less for technical goods!) deliver with the German post parcel system ("DHL"), and if they do, you'll get some "sms" on your (German!) mobile (but you don't need it to work 365/365, just when you're awaiting some parcel) with a 4-digit code by which, together with your "Gold Card", you then can fetch your parcel during the next 8 days (but not a day longer) from such a delivery station, for most of them 24 hours a day incl. week-ends, the interest laying in German sellers-by-mail being LOTS cheaper than, say, French ones, most of the time.

On the other hand, OT again, I would never buy a German pc anymore: Just lately, my try to install some MS sw update ("Service Pack") failed miserably, even after my trying to apply all sorts of MS "registry checking/repairing" tools recommended within the web, and for the (by some threads over there) presumed reason that MS simply isn't able to hold every single service pack and such compliant to any language world-wide, so that updates to the (native) English versions work fine, whilst more often then not, even updates to English MS sw, but installed on a "foreign" Windows (here: German pc's with German Win), don't update properly, or don't update at all - which means you'll need, to be safe on this account at least, English Win, and English applications, wherever you live.

Thus, my future pc's will come from GB, exclusively (and hoping there will not be any problems caused by "Win U.S. vs. Win GB"), even if that complicates possible guarantee issues (well, at the end of the day, either your hardware works over many years, or falls flat 1 month after guarantee period expiring, by "planned obsolescence", so this doesn't make too much difference). Price level in France, e.g., is exceptionally high, so that even with additional postage costs, buying in GB, from a French's pov, is almost always a very good idea, both with regards to technical stuff and with books and "consumer media" (film DVD's even with French soundtrack, etc.).

Of course, we all know original U.S. prices are about half-price, but with transportation costs, and then severe European taxes, original price is doubled, and if something goes wrong, exchange/repair costs (and custom probs) quickly skyrock, not speaking of the strain on your patience... Unfortunately, I cannot give any hint how to really smoothen out delivery to the Continent, either from the U.S. or just from GB (where it's much easier but of much less interest, comparatively); there are some remailers but who are so expensive (and they don't make you avoid custom) that their only possible interest lies in getting goods from U.S. sellers who refuse abroad deliveries.

Oh, there are two really good ways indeed:

- Have an American pen pal; this might even be a way to avoid European customs and taxes, depending on various factors

- Get acquainted (or in love) with a U.S. soldier on the Continent: He'll get almost everything for almost free, and you will, too... and this brings us to this thread's shared find: Manuel Pradal's 1997 film "Marie Baie des Anges" (but beware, it's just for real hardcore Frenchies; btw, did I ever speak to you of Edouard Nierman's 1987 movie Poussière d'Ange, or then, of Christophe Ruggia's Les Diables (2002)? Well, that's very particular, as is Terrence Malick on the bright shore of the Ocean. ;-)