Impact of Surveillance on web development

Post here if your message doesn't fit into another forum but is still about web development. Includes site critiques, web hosting and server questions, helpful software and resources, and more.
Post Reply
User avatar
MikeGale
Rank VI - Professional
Rank VI - Professional
Posts: 699
Joined: Mon Dec 13, 2004 1:50 pm
Location: Tannhauser Gate

Impact of Surveillance on web development

Post by MikeGale » Thu Nov 14, 2013 4:37 pm

I was reading an interesting article today, http://j.mp/1eREyq6.

It's about how the encryption on an SSL link is quite complex. (45 different combinations when I checked a Win 7 machine, same on Win 8.1)

The technology enables every user to have some say in how securely he connects over a secure connection. (The person who configures the web server also has some control.)

I'm not sure how this ties in with development and validation. What are your thoughts?

Neither am I sure how this plays out across different browsers.

It is entirely possible, that a sizeable proportion of users will start adjusting these settings. (It's not trivial to do. Wouldn't be surprised if people start writing little utilities to do it for you.) If it has an impact it would be good to be prepared. (To do it yourself see an article linked to that above.)

As this negotiation (for the cypher suite) is hidden its easy to have no idea it's going on, so worth knowing about.

Here's some of those suites:
  1. TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P256
  2. TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P384
  3. TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P521
  4. TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA_P256
  5. TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA_P384
  6. TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA_P521
  7. TLS_DHE_DSS_WITH_AES_256_CBC_SHA
  8. TLS_DHE_DSS_WITH_3DES_EDE_CBC_SHA
  9. TLS_RSA_WITH_AES_256_CBC_SHA
  10. TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA_P256
  11. TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA_P384
  12. TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA_P521
  13. TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA_P256
  14. TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA_P384
  15. TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA_P521
  16. TLS_DHE_DSS_WITH_AES_128_CBC_SHA
  17. TLS_RSA_WITH_AES_128_CBC_SHA
  18. TLS_RSA_WITH_RC4_128_SHA
  19. TLS_RSA_WITH_3DES_EDE_CBC_SHA
  20. TLS_RSA_WITH_RC4_128_MD5
  21. SSL_CK_RC4_128_WITH_MD5
  22. SSL_CK_DES_192_EDE3_CBC_WITH_MD5
  23. TLS_RSA_WITH_NULL_SHA
  24. TLS_RSA_WITH_NULL_MD5
  25. TLS_RSA_WITH_AES_128_CBC_SHA256
  26. TLS_RSA_WITH_AES_256_CBC_SHA256
  27. TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256
  28. TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384
  29. TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P521
  30. TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256_P256
  31. TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384_P384
  32. TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384_P521
  33. TLS_DHE_DSS_WITH_AES_256_CBC_SHA256
  34. TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256
  35. TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P384
  36. TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P521
  37. TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256_P384
  38. TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256_P521
  39. TLS_DHE_DSS_WITH_AES_128_CBC_SHA256
  40. TLS_RSA_WITH_NULL_SHA256
  41. TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384_P384
  42. TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384_P521
  43. TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256_P256
  44. TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256_P384
  45. TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256_P521
Looks like complexity run a bit wild, to me!

User avatar
Albert Wiersch
Site Admin
Site Admin
Posts: 3236
Joined: Sat Dec 11, 2004 9:23 am
Location: Near Dallas, TX
Contact:

Re: Impact of Surveillance on web development

Post by Albert Wiersch » Fri Nov 15, 2013 5:30 pm

MikeGale wrote:I'm not sure how this ties in with development and validation. What are your thoughts?
Hi Mike,

That was a good article. I liked it.

As for HTML and CSS validation, I don't see how it ties in... but for running servers and for businesses demanding more security than the standard stuff, it definitely applies.

I can see how a financial institution might want to enforce a stronger “Cipher Suite”.

I can also envision how some developer tools that support HTTPS might also provide some insight into security, so in that sense a program like CSE HTML Validator, when using the Batch Wizard to check HTTPS links, might report back on security issues, or perhaps could be used to test secure communications with a server that limits the Cipher Suites.
Image
Albert Wiersch

User avatar
MikeGale
Rank VI - Professional
Rank VI - Professional
Posts: 699
Joined: Mon Dec 13, 2004 1:50 pm
Location: Tannhauser Gate

Re: Impact of Surveillance on web development

Post by MikeGale » Fri Nov 15, 2013 10:48 pm

Agreed no direct impact on HTML and CSS validation that comes to my mind.

One impact that did occur to me is people who are optimising page load times. Fairly closely related to validation. These settings (browser and server both) might have noticeable impacts, in some circumstances. (Might be mentioned in messages, if there's some research work showing how much the impact can be!)

Agreed there is room for useful reportback in Batch Wizard. (It does mean testing real servers rather than staging machines!) Would be useful to know that a server doesn't do perfect forward security. Given the current attention to surveillance, this is distinctly useful.

<JustABrainstormIdea>That triggers a thought in my head. It would be possible to emit requests to a server with restricted suite lists. By sending several of those interesting findings could result. (Even wilder idea, also link that into controlling how the server responds...)</JustABrainstormIdea>

User avatar
Albert Wiersch
Site Admin
Site Admin
Posts: 3236
Joined: Sat Dec 11, 2004 9:23 am
Location: Near Dallas, TX
Contact:

Re: Impact of Surveillance on web development

Post by Albert Wiersch » Mon Nov 18, 2013 9:58 am

Hi Mike,

Yes, some interesting ideas for the Batch Wizard. I am not familiar with what the component I use can do, but it could probably do some interesting things, like make a request with a restricted cipher suite list... but currently I just don't see a lot of demand for this. Definitely something to keep in mind though.

Do you have any actual need/want of anything like this or is this just "interesting" to think about and ponder?

And for the cases where one would want or need such security testing, they'd probably want something more dedicated to it with a lot of security functionality.
Image
Albert Wiersch

User avatar
MikeGale
Rank VI - Professional
Rank VI - Professional
Posts: 699
Joined: Mon Dec 13, 2004 1:50 pm
Location: Tannhauser Gate

Re: Impact of Surveillance on web development

Post by MikeGale » Mon Nov 18, 2013 2:36 pm

Hi Albert,

I'm sure I'm not alone in being deeply perturbed about the sheer magnitude of the spying on the Internet. I knew things were going on but the ongoing revelations drive home how pervasive it is.

Like others it makes me consider turning my back on a system that has gone wrong. That's impossible.

I'm still thinking about where this takes me and everybody.

I'm not alone in this and regularly see new things being talked about by growing numbers of people. I see a new VPN system each week, the debate about how to make all browser traffic more secure (3 choices being bandied around in IETF), now Indonesian politicians are going nuts...

Now Yahoo announces HTTPS on everything.

The coverage is deeply biased and extremely misleading. It's all about the Five Eyes spies, it ignores the other spy agencies, the Internet advertisers watching many keystrokes of millions of people ... So the problem is worse than the mass media seem aware of.

There was no real investigative journalism here, now the clamps (against leakage) are well and truly on, good chance it can get way worse if it's not fixed now. The press is no safeguard in future, they just got a windfall. I think enough people realise that this is a maybe once only chance to sort things out.

I think there's a groundswell here. Part of the reason the surveillance community is kicking out stuff in an attempt to justify themselves. (Blocking porn search terms...)

The world may be undergoing a paradigm shift.

So, no I haven't decided what I myself need. Making Internet communication more secure is now unavoidable. How this fits with CSE? I don't have a clear suggestion, yet. (So it is indeed a thought experiment at present!)

User avatar
Albert Wiersch
Site Admin
Site Admin
Posts: 3236
Joined: Sat Dec 11, 2004 9:23 am
Location: Near Dallas, TX
Contact:

Re: Impact of Surveillance on web development

Post by Albert Wiersch » Tue Nov 19, 2013 8:39 am

It no doubt an interesting world we live in.

Personally, it's sort of like spam to me. Years ago, I would get angry when I got spam and would often times try to report it or do something about it, but it's gotten to the point where I expect it as a part of life and I just delete it without getting mad anymore. It's come to the point where I just expect to be tracked and watched. With all the complexity of the modern world and smartphones that can track you with GPS, it seems rather futile, to me anyway, to expect privacy if you want to live in the modern world and be connected.

I am all for better & stronger HTTPS and other encryption, but my main fear is cyber theft. It seems all too easy to for a crook to get a mass of user data from breaking into systems or from social engineering. The stolen data can then be used for identity theft. Some of it isn't too bad (just a minor hassle), like credit card theft that you can just report to your credit card company/bank and get a new card while having the fraudulent charges removed, but when they get your social security number and take out loans in your name and file false tax returns, then it can become a huge hassle.

I'm not sure what it is like in Australia, but in the US it seems like more and more people are having to go through these huge hassles to clear up fake loans and fake tax returns when a criminal uses their private info to commit fraud... I think some people have even had their homes stolen from them (rare, but criminals can do that too easily by posing as you and signing fake documents in your name). Recently in the US, a couple of convicted murderers got out of prison by using forged documents with bogus signatures of prosecutors and judges (probably "copy and pasted").

I don't think strong encryption can do much to solve the above, because it has more to do with lax security practices that go far beyond encrypting transmissions. I wish companies would focus more on those problems and criminals stealing their data.

Perhaps a bit off-topic, but an interesting subject. :D
Image
Albert Wiersch

User avatar
Lou
Rank V - Professional
Rank V - Professional
Posts: 246
Joined: Fri Jul 29, 2005 5:55 pm
Location: CO
Contact:

Re: Impact of Surveillance on web development

Post by Lou » Tue Nov 19, 2013 11:45 am

Right on target (topic) I thank. An article that points to where education could be focused is http://www.datagenetics.com/blog/september32012/ The first grabber point is that only 20 of the possible 10,000 4-digit pin numbers are 26% of the pins used! The pin '1234' is almost 11% of the pins studied.

With that type of "security" who needs NSA? I'm sure everyone has heard the quip that 'NSA doesn't know anything you haven't already posted on facebook.'

As you said Albert, not much point in getting mad. When I first was "forced" into a facebook account I got a chuckle out of the pervert facebook though I was. At the time I was working with some junior high school boys doing theatre lights and sound. Based on my age (which was privet) the facebook ads were for Viagra. Based on other friends, suggested 'friends' were all jail-bait! (As for my age, I found out Kennedy had been shot when I went to the college register's office to drop a course and they were watching TV.) I'm sure we all have noticed the pointed ads that seem to pop up every where. A month ago I needed to find some glycol to winterize my house. Ever sense I keep seeing ads for a hydrometers.

I don't mean to justify or condone over reach, but I think that much of the concerns being voiced are due to the change in the public perception of the Government. Currently, I am hearing that this change in perception started with the conspiracy theories around Kennedy and the Warn Commission.

In the spirit of openness, my military and most of my civilian careers revolved around communication and computer security.
Lou
Say what you will about Sisyphus. He always has work.

User avatar
MikeGale
Rank VI - Professional
Rank VI - Professional
Posts: 699
Joined: Mon Dec 13, 2004 1:50 pm
Location: Tannhauser Gate

Re: Impact of Surveillance on web development

Post by MikeGale » Tue Nov 19, 2013 3:34 pm

Even though this seems off topic, on the surface, I don't think it is. It's part of the background to the web and is already changing technologies and killing companies. The reach of the ideas will be wider in future. A couple of points:
  1. On spam. I used to report much of the spam I got. Then the you-can-spam act was passed by ignorant politicians and the gates opened (within about 2 weeks). I gave up. That spam is from a range of criminals and people with "thought diseases" who are not tax payer funded. These spies are not like that. They're taxpayer funded and should be prevented from over-reach by those taxpayers. (In my view it goes wider than that. It's not just the taxpayers of your own country that deserve respect...)
  2. In a situation where they won't tell you anything, and where the law has been perverted to force corporations to lie, it's prudent to assume things.
    1. I already know that this network is used officially to aid and abet US companies who make significant political contributions.
    2. I also know that Snowden gained access to a very wide range of information. Snowden worked for a company that is now part of the a commercial group that has wide interests, including computer technology, retail, manufacturing... There is no known way to prove that information hasn't been similarly taken for less public spirited purposes and funnelled to such companies...
    3. The NSA, GCHQ, PLA Unit 61398, the FSB... have a need for ideas, and code to better spy.
    4. In my view it's prudent to assume that somebody has gained commercial and technical advantage by misusing state and commercially controlled surveillance. So if you have a really worthwhile idea that you discuss here and in email (say something started in this forum), it's prudent to imagine that idea being presented at a daily briefing in Fort Meade, Shanghai, Moscow with 24 hours. For some people that's very big news. When even your private networks that you pay a lot of money for are routinely breached in this way... This fundamentally alters the landscape of thought.
  3. It's plain that various things are not fit for purpose in the current world. eMail protocols, content transfer protocols (http!), the way security certificates (SSL...) exist, elected representative oversight and lawmaking, the judiciary... These need to change and that has started. I've seen apathy swept aside as people realise the personal impact. They realise that there's people out there maybe watching their every move, with impunity. These systems have started evolving. A couple of people, including some ex Seals and SAS, are having a shot at fixing email. The IETF group that is looking to the next version of HTTP has gained more urgency. Fixing your crypto suite profile, using tor, vpn's... This has happened in a short time. (Now we just need leaks from a few more spy agencies and commercial surveillance operations to help the public realise this is a wider problem. And lay off the NSA etc. to some extent!)
Agreed Albert that just some of these things don't fix the problem entirely. I believe that they are part of the solution. If we, say, had better secured email this would reduce the number of clues used by the bad guys to steal identities. Then you need more. <Brainstorm>i.e. if fora (forums?) had a small http accessible part and the bulk happened in a secured HTTPS area that would make a difference too</Brainstorm>

Agreed Lou. There is personal responsibility here. Many who were apathetic are going to get temporary motivation, I guess, then relapse. Those who are prepared to persistently give it attention will get better results if their efforts are not undermined by a shoddy infrastructure.

I expect to see growing impact.

User avatar
MikeGale
Rank VI - Professional
Rank VI - Professional
Posts: 699
Joined: Mon Dec 13, 2004 1:50 pm
Location: Tannhauser Gate

Re: Impact of Surveillance on web development

Post by MikeGale » Wed Nov 20, 2013 12:41 am

Here's a couple of articles that suggest that the DNS system is fragile and may have been used for nefarious purposes:

http://j.mp/1h1VKtL

http://j.mp/1bCemJD

User avatar
Albert Wiersch
Site Admin
Site Admin
Posts: 3236
Joined: Sat Dec 11, 2004 9:23 am
Location: Near Dallas, TX
Contact:

Re: Impact of Surveillance on web development

Post by Albert Wiersch » Thu Nov 21, 2013 7:42 am

MikeGale wrote:Here's a couple of articles that suggest that the DNS system is fragile and may have been used for nefarious purposes:

http://j.mp/1h1VKtL

http://j.mp/1bCemJD
Thank Mike. That was indeed more interesting reading. I have to wonder if it would be possible for some sort of an attack to take down the Internet by flooding it with fake routes.

In this day & age, anything relying on the "honor system" needs to be fixed as its far too vulnerable to those who would abuse it.
Image
Albert Wiersch

User avatar
MikeGale
Rank VI - Professional
Rank VI - Professional
Posts: 699
Joined: Mon Dec 13, 2004 1:50 pm
Location: Tannhauser Gate

Re: Impact of Surveillance on web development

Post by MikeGale » Thu Nov 21, 2013 4:21 pm

Much of what I've known about that is "unloaded from RAM" at the moment so can't comment thoroughly.

I imagine that a route that doesn't deliver anything, normally gets sorted out pretty quickly.

Where something is delivered (like these cases) the people responsible need the capacity to handle the traffic. That eliminates a lot of potential perps. and also means that those who do it have an installed capital base that they would, hopefully, not want to put at risk!

Some people already have measures to stop this in place, if I read things right. I also think that Renesys have developed their own way to sort it out, which they're offering to their customers.

The Internet is pretty self healing, despite the fact that undersea cables and Internet Exchanges are undesirable choke points. Renesys has documented some interesting incidents, like cable cuts off the Egyptian coast where the slack can now be taken up by routes through Iran/Russia... The healing tends to happen extremely quickly. (No doubt has big impacts on who pays transit to whom!)

I imagine that a minor diversion, with a plausible path, might go unnoticed. Now that is truly scary.

User avatar
MikeGale
Rank VI - Professional
Rank VI - Professional
Posts: 699
Joined: Mon Dec 13, 2004 1:50 pm
Location: Tannhauser Gate

Re: Impact of Surveillance on web development

Post by MikeGale » Thu Jan 30, 2014 1:32 pm

Thanks for noting that book.

It looks like a good antidote to the highly selective and sometimes hysterical press coverage.

I see there's a new edition, with a name change, Glass Houses: Privacy, Secrecy, and Cyber Insecurity in a Transparent World. (Aug 27, 2013)

I've made a note of it.

My take is that government, law and the media are unable to deal with the world we live in, adequately. One solution would be to bring some of the smart guys out from the securocracy, remind them of the 4th amendment etc. and set them onto helping find a solution. I think any solution of value needs to transcend national boundaries (which brings with it much risk).

This is kinda what's been going on, but we need a solution without the reckless disdain for the man in the street, and the company in the street, shown in recent years. Heck in at least one slide show one of these guys refers to the man in the street as the adversary. Complete nuttiness, like that, needs to be fixed.

gmgj
Rank 0 - Newcomer
Rank 0 - Newcomer
Posts: 1
Joined: Tue Feb 09, 2016 10:59 am

Re: Impact of Surveillance on web development

Post by gmgj » Tue Feb 09, 2016 12:35 pm

I strongly agree with:

"I'm sure I'm not alone in being deeply perturbed about the sheer magnitude of the spying on the Internet. I knew things were going on but the ongoing revelations drive home how pervasive it is."

I thought I would use personal examples to show my concern.

Tracking, surveillance by another name.

I browse for something on MON, and see related ads for the whole week. Do you know what the smallest book in the world is? "The conscience of marketer".

There is a new browser, https://www.brave.com/. "Brave keeps you and your information safer, effectively shielding you from 3rd party tracking and malvertisement."

On the server side, I have a vanity site, 90 percent of the hits to my site come from the xyz domains submitting bad uris.
I changed my 403 message to try and discourage this.

<!--#if expr="(${REQUEST_URI} = /share-buttons/) ||
(${REQUEST_URI} = /free-traffic/) ||
(${REQUEST_URI} = /traffic2cash/) ||
(${REQUEST_URI} = /social-widget/) ||
(${REQUEST_URI} = /net-profits/)"-->
Please go Away and stay Away.
http://garyjohnsoninfo.info/?from=http: ... ttons.xyz/
http://garyjohnsoninfo.info/?from=http: ... affic.xyz/
http://garyjohnsoninfo.info/?from=http: ... 2cash.xyz/
http://garyjohnsoninfo.info/?from=http: ... ofits.xyz/
http://garyjohnsoninfo.info/?from=http: ... idget.xyz/

No effect. I would be nice if I could contact the person who owns these domains. Yes it would.

Post Reply