Monday, November 15, 2010

Distributed Privacy : A Distraction

I was reading the recent Register article on Privacy legislation and it struck an interesting note with me. It assumes that data collection is inevitable, and the only way to police it is through "do not collect" or "do not follow" tags.

That's problematic for a whole heap of reasons, some of which the article author outlines (How do you know not to store someones details in the future without storing their details), but it all stems from the same point. Data collection is inevitable. Data ends up in a multitude of tables across the internet, and it is impossible to know who knows what about you, especially as people sell that information to others.

Its a familiar problem to the UK government. Concerns about the NHS databases, ContactPoint, Offender Registers, etc have gotten much more traction in the UK media than traditional "Big DB" companies like Experia who store / sell your credit report and is pretty much vital to anyone needing credit.

The answer to this is allowing people to store their own information, or in the case of dependents have it managed for them. Each person has a dedicated store of their information, demographic, medical, financial, friends, contacts, etc. Each person can choose how much information is given out and to whom and on what basis. The responsibility for the data becomes that of the person, not of the data collectors. Services then pull this information together however they need to show it.

Take Facebook friends lists as an example.

I have a list of IDs who I am friends with, they have list of people they are friends with, and so on, making a large graph of a chunk of the planet. With the current system, Facebook has all this information, pulls back the list of my friends, maps those IDs to some people and shows them to you, whether my friends want this or not.

Instead, lets take a distributed privacy approach to this instead. Facebook now requests my list of friends. Immediately I've got control. I've got 400 friends, however I don't want anyone to currently know about 20 of them, my store therefore only returns the 380 friends I'm allowing people to see.

These are only IDs as well, there's no personally identifiable information in there. Facebook now needs to request the relevant information from each of the IDs to show my friends list. However out of the 380 people that I'm showing as friends, 80 of them don't want to share their details right now. Facebook then shows my 300 friends that it knows about (and might say, and 80 people that we don't know about).

That's my privacy respected, and your privacy respected right?

That works from the FB perspective, now lets take a look at something like the massive ContactPoint database which was to store lots of information about the children of Britain.

There is plenty of sensitive data involved with children, and its always an evocative subject. The key worries were over the what data was stored, how to change that information, and who had access to it (The last one being the big media "worry"). The list of people who had access to the db went from a massive amount of people pre-outcry to such a small amount of people post-outcry that it was not worth running the DB in the first place.

Lets concentrate on the last part. Who has access to it. How can we allow GPs, social services, the police, clinical psychologists all access your child's record without allowing your next door neighbor who works at the local health center access?

By the data being distributed and owned by the person, we can also introduce the idea of "Sensitive access". Any data deemed to be sensitive requires the person to authorize its release from their data store. This data could be different things to different people. Demographic information for a person running from an abusive spouse for example, or the medical record of a child.

It would work pretty simply. The digital signature of the person making the request is sent along with the request. If the data being asked for is flagged as being sensitive, a confirmation for its release is sent to the person. If its the GP requesting access you give it; the next door neighbor is denied (and potentially reported).

There are more benefits here, I can see all data that's being held on me, as I'm the one that's holding it. If its wrong, I can change or delete it (see below). I'm ultimately the one responsible for my data. No more data breaches, no more records being left on the train / bus stop, its my data.

Changing and deleting records are a fun thing. Obviously, I want to be able to change my clinical record if its wrong (No, I'm allergic to penicillin not pencils!), but how about negative features like criminal record? Obviously, if we want to keep our idea pure, things like your criminal record should be part of the whole data set that you own. However should you be allowed to delete your criminal record, and act like it never happened? Same goes with pretty much anything required by the government; tax records, National Insurance number, Birth Certificate, etc.

This is the only place where the idea gets awkward. Its also the place where a lot of the privacy ("I don't want the government poking into my affairs") purists will disagree with me. This data should be shared between yourself and the government. It can't be changed *by either party* without both sides consenting. That's quite an important distinction. It means that I can't change my negative elements (say, wiping out the points on my driving license) without the shared source agreeing to it. Likewise, they could not add to it (adding points onto my license) without my agreement, which may have to be court ordered if I really don't agree.

There are a few safeguards that would have to be put in place. Guardianship transfer would be important. Taking control of an account away from an abusive parent / guardian would be vital to maintain the security of the individual. There would also have to be a security related aspect to this, as having the Intelligence services writing their security reports onto the record of an individual suspected of terrorism would be hilarious.

The last part of this is enforcement. It would have to be illegal for people to track you anymore, or this becomes a moot point, and the big players continue invading everyone's privacy as usual. Big databases full of your information owned by other people would have to be made illegal. There would need to be processes for working out disputes of shared information. There would still be the argument about what data is owned by who ( Are the headers of my Email messages owned by me? How about the data ISPs use for traffic shaping?).

Lastly, this is such a sea change, that would effect so many big established players, that would take a massive effort on the part of both governments and their people, a huge amount of storage and servers to host all the information (and who owns / runs the servers your data is on? Do you trust them?), that this is effectively a pipe dream... Maybe the Register's unspoken assumption is correct and the battle for data privacy is already lost.

More things to consider

  • Responsibility for the data being correct now lies with the individual. If the data you use is wrong, its the person's fault not yours. This means a massive reduction in red tape, lawsuits, and pretty much anything connected with the storage of data.
  • We are pretty much killing, Google, Facebook, many advertisers, and anyone else who relies on having a big database full of your information that it can use to track you or sell.
  • People would have to be educated as to the importance of security themselves. If their account gets hacked, they are stuffed... but it would be their fault.

Friday, August 27, 2010

Part 1.3 Getting ANT up and running

We are now going to compile our little Hello World application. Normally we'd be using Flex Builder for this, choosing build from the context menu or CTRL-B-ing our way to some fun. Remember however that we don't want to use FB for compilation so we are going for a proper build system approach.
On the surface this seems like a lot of work for something FB can achieve for us very simply, but as we'll discover the further we get through these tutorials, it becomes a very powerful tool in our hands. Hopefully you'll use these powers for good rather than evil...
I'm actually going to provide the build file this time, and then we are going to work our way through it. I'm not going to provide a full set of learning ANT documentation, there is plenty of that out there if you are interested.


  • Turn off every "build before..." setting you can find in FB, they slow things down and through off what we want to deploy!
  • Add another folder to your setup folder called build, and under that add a file called build.xml.
  • Make sure the FLEX_HOME environment variable is set to the Flex SDK you want to use.
  • Grab the Flex Tasks jar file from your SDK folder and place it into the libs directory of your project.
Here's the code we're going to be adding to the build file. This is a pretty simple file, and doesn't include many if any of the optimizations and tweaks that ANT provides.

We now need to get this running.


  • Open the ANT view in Flex Builder. You should have this already installed if you've followed the previous parts of the tutorial.
  • Choose to add a buildfile and choose the projects buildfile
We should see the below.
Double click the compile target, and assuming everything has been set up properly, the SWF will be generated in the target directory.
We can clean the folder structure back up by running the clean task, ready to be committed to source control once we are satisfied that the code works.

Part 1.2 Project Structure.

First of all create a new Flex LIBRARY project. Though we are going to be generating swfs, we aren't going to be having Flex Builder do it.

Why? Well we aren't going to be the only ones creating the swfs. In a properly ordered world, we developers only build our code to test it, make sure it's working, compiling, and then we leave it. If we want to run a Continuous Integration process for example, we need to be building the artifacts in some other way than FB.


Now create the folder structure the same as the image. I'll explain why we have all these top level folders.
  • conf - Short for configuration. This is where we are going to add the files that we use to dynamically configure the application at runtime. We keep these separate from the source files deliberately. These configuration files are able to be changed without (necessarily) recompiling the application.
  • libs - We keep all the libraries for the project. We will have a combination of Flex SWCs and Java JARs in here, split up by language.
  • setup - We are going to be keeping our build files in here along with any other files we use to generate artifacts. This might be additional config files for the Flex compiler for example.
  • src - Our main source directory. All our source code which will end up in production will end up in here.
  • target - This directory can be removed later on as we will be creating it on the fly. This is where all the compiled artifacts end up, this includes production SWFs, SWCs or Test SWFs.
  • test - Where all our test code is going. This includes mocks and additional files in order to create the unit tests.
  • util - This is where all the useful bits and pieces go that don't fit in elsewhere. Server start up scripts, Value Object generation scripts( like xDoclet ), etc.
In preparation for the next part of the tutorial, we're going to add some additional files.
  • Under source, add the following chain of folders com / codecradle / tutorial / application / view
  • In this final folder add a simple HelloWorld MXML application (Hint, don't choose new MXML Application under the context menu, choose MXML component and then set up the application manually. If you choose the MXML Application, FB tries to do all sorts of clever things we don't want it to do). We'll call this something nice and imaginative... like Tutorial.mxml ;)
  • Change the output folder in FB to "target" rather than "bin" just so FB doesn't keep trying to create it.
So by the end of this part, we should have something like the image below. Like all the other parts of this tutorial, add a comment if you have any questions!

More Flex : Hero information comes out

Deepa has furnished us with some more Hero related snippets, and they make some interesting reading.

Deepa's posting

The first spec of the Spark Datagrid makes an appearance, which as suspected, was closer to the Halo Datagrid rather than the AdvancedDataGrid. *makes sign to ward off evil*

There's some good reading in the rest of the spec's that have been posted up to the Open SDK site. The size report looks like it might be useful pre-production, and the RSL Enhancements is worth a read just to get more information on how the Flash Player handles loading of RSLs generally.

Other than this, we still need to wait to get a proper nightly build. Deepa's implication is that the impetus for this was on the Player team rather than Flex SDK team to get their stuff done (or maybe I'm reading too much into it ;) ) as Hero is closely tied to the new player features.

Lastly, it looks like Hero is semi-officially Flex 4.5 rather than 5. Rumours as to what will be in 5.0 starting in 3... 2... 1...

Wednesday, July 28, 2010

Stop using Maven for open source projects... please...

Honestly, just stop it.

Locking us into a particular build system is simply wrong. Say if a particular popular open source project uses Maven and I want to integrate it into a build system based off of ANT? No luck without a stupid amount of work.

Want to find the actual library a project is using so you can root around inside it? Good luck with that.

Compatibility issues between different versions means that I have to wipe my local repository when switching between two large projects (which I do frequently)... How is this saving me time again?

I have no problem bundling a build system with a project (it would be stupid not to) but Maven isn't just a build system, it's a dependency management system as well. If I was frothing at the mouth more, I'd call it a dependency obfuscation system...

Please, please, please... and I'm looking squarely at you Apache. If you must use Maven, include an alternative method of building the project for the rest of us.

Thursday, July 1, 2010

Bye Bye Slider, Hello Hero!

So Slider, the mobile version of the Flex framework is dead.

Instead, we are getting Hero, the code name for the new Flex framework. No word as yet whether this is a point release or Flex 5...

Deepa threw a whole load of links into her blog post about it. Most pertinent being...

http://opensource.adobe.com/wiki/display/flexsdk/Hero
http://labs.adobe.com/technologies/flex/mobile/faq.html

Most interesting part of this is finally a major update to the Datagrid! Anyone who's been kicking around from Flex will remember the last go at updating the Datagrid. The horrid AdvancedDataGrid which, and lets be fair to it here, was barely usable...

Once the news gets out in a proper way, I'd imagine the SDK team will be inundated with kitchen sink requests for a new Datagrid.

Which will we get from the Spark Datagrid... A decent replacement or ADG 2.0?

Monday, May 10, 2010

Part 1.1 Setting up the project.

Ok, so first things first we need to set up Flex Builder and ANT. There are many many blogs out there for doing this, so I'll not going to add to the noise. Instead, some handy links.
http://www.mobilefish.com/developer/flex/flex.html
http://www.roseindia.net/flex/flex_builder_ide.shtml
http://www.judahfrangipane.com/blog/?p=193
http://blog.jodybrewster.net/2008/04/09/installing-ant-in-flex-builder-3/

Go Ahead, get all that sorted out. Part 2 will be creating the project and how we are structuring it.