Monday, November 15, 2010

Distributed Privacy : A Distraction

I was reading the recent Register article on Privacy legislation and it struck an interesting note with me. It assumes that data collection is inevitable, and the only way to police it is through "do not collect" or "do not follow" tags.

That's problematic for a whole heap of reasons, some of which the article author outlines (How do you know not to store someones details in the future without storing their details), but it all stems from the same point. Data collection is inevitable. Data ends up in a multitude of tables across the internet, and it is impossible to know who knows what about you, especially as people sell that information to others.

Its a familiar problem to the UK government. Concerns about the NHS databases, ContactPoint, Offender Registers, etc have gotten much more traction in the UK media than traditional "Big DB" companies like Experia who store / sell your credit report and is pretty much vital to anyone needing credit.

The answer to this is allowing people to store their own information, or in the case of dependents have it managed for them. Each person has a dedicated store of their information, demographic, medical, financial, friends, contacts, etc. Each person can choose how much information is given out and to whom and on what basis. The responsibility for the data becomes that of the person, not of the data collectors. Services then pull this information together however they need to show it.

Take Facebook friends lists as an example.

I have a list of IDs who I am friends with, they have list of people they are friends with, and so on, making a large graph of a chunk of the planet. With the current system, Facebook has all this information, pulls back the list of my friends, maps those IDs to some people and shows them to you, whether my friends want this or not.

Instead, lets take a distributed privacy approach to this instead. Facebook now requests my list of friends. Immediately I've got control. I've got 400 friends, however I don't want anyone to currently know about 20 of them, my store therefore only returns the 380 friends I'm allowing people to see.

These are only IDs as well, there's no personally identifiable information in there. Facebook now needs to request the relevant information from each of the IDs to show my friends list. However out of the 380 people that I'm showing as friends, 80 of them don't want to share their details right now. Facebook then shows my 300 friends that it knows about (and might say, and 80 people that we don't know about).

That's my privacy respected, and your privacy respected right?

That works from the FB perspective, now lets take a look at something like the massive ContactPoint database which was to store lots of information about the children of Britain.

There is plenty of sensitive data involved with children, and its always an evocative subject. The key worries were over the what data was stored, how to change that information, and who had access to it (The last one being the big media "worry"). The list of people who had access to the db went from a massive amount of people pre-outcry to such a small amount of people post-outcry that it was not worth running the DB in the first place.

Lets concentrate on the last part. Who has access to it. How can we allow GPs, social services, the police, clinical psychologists all access your child's record without allowing your next door neighbor who works at the local health center access?

By the data being distributed and owned by the person, we can also introduce the idea of "Sensitive access". Any data deemed to be sensitive requires the person to authorize its release from their data store. This data could be different things to different people. Demographic information for a person running from an abusive spouse for example, or the medical record of a child.

It would work pretty simply. The digital signature of the person making the request is sent along with the request. If the data being asked for is flagged as being sensitive, a confirmation for its release is sent to the person. If its the GP requesting access you give it; the next door neighbor is denied (and potentially reported).

There are more benefits here, I can see all data that's being held on me, as I'm the one that's holding it. If its wrong, I can change or delete it (see below). I'm ultimately the one responsible for my data. No more data breaches, no more records being left on the train / bus stop, its my data.

Changing and deleting records are a fun thing. Obviously, I want to be able to change my clinical record if its wrong (No, I'm allergic to penicillin not pencils!), but how about negative features like criminal record? Obviously, if we want to keep our idea pure, things like your criminal record should be part of the whole data set that you own. However should you be allowed to delete your criminal record, and act like it never happened? Same goes with pretty much anything required by the government; tax records, National Insurance number, Birth Certificate, etc.

This is the only place where the idea gets awkward. Its also the place where a lot of the privacy ("I don't want the government poking into my affairs") purists will disagree with me. This data should be shared between yourself and the government. It can't be changed *by either party* without both sides consenting. That's quite an important distinction. It means that I can't change my negative elements (say, wiping out the points on my driving license) without the shared source agreeing to it. Likewise, they could not add to it (adding points onto my license) without my agreement, which may have to be court ordered if I really don't agree.

There are a few safeguards that would have to be put in place. Guardianship transfer would be important. Taking control of an account away from an abusive parent / guardian would be vital to maintain the security of the individual. There would also have to be a security related aspect to this, as having the Intelligence services writing their security reports onto the record of an individual suspected of terrorism would be hilarious.

The last part of this is enforcement. It would have to be illegal for people to track you anymore, or this becomes a moot point, and the big players continue invading everyone's privacy as usual. Big databases full of your information owned by other people would have to be made illegal. There would need to be processes for working out disputes of shared information. There would still be the argument about what data is owned by who ( Are the headers of my Email messages owned by me? How about the data ISPs use for traffic shaping?).

Lastly, this is such a sea change, that would effect so many big established players, that would take a massive effort on the part of both governments and their people, a huge amount of storage and servers to host all the information (and who owns / runs the servers your data is on? Do you trust them?), that this is effectively a pipe dream... Maybe the Register's unspoken assumption is correct and the battle for data privacy is already lost.

More things to consider

  • Responsibility for the data being correct now lies with the individual. If the data you use is wrong, its the person's fault not yours. This means a massive reduction in red tape, lawsuits, and pretty much anything connected with the storage of data.
  • We are pretty much killing, Google, Facebook, many advertisers, and anyone else who relies on having a big database full of your information that it can use to track you or sell.
  • People would have to be educated as to the importance of security themselves. If their account gets hacked, they are stuffed... but it would be their fault.

Friday, August 27, 2010

Part 1.3 Getting ANT up and running

We are now going to compile our little Hello World application. Normally we'd be using Flex Builder for this, choosing build from the context menu or CTRL-B-ing our way to some fun. Remember however that we don't want to use FB for compilation so we are going for a proper build system approach.
On the surface this seems like a lot of work for something FB can achieve for us very simply, but as we'll discover the further we get through these tutorials, it becomes a very powerful tool in our hands. Hopefully you'll use these powers for good rather than evil...
I'm actually going to provide the build file this time, and then we are going to work our way through it. I'm not going to provide a full set of learning ANT documentation, there is plenty of that out there if you are interested.


  • Turn off every "build before..." setting you can find in FB, they slow things down and through off what we want to deploy!
  • Add another folder to your setup folder called build, and under that add a file called build.xml.
  • Make sure the FLEX_HOME environment variable is set to the Flex SDK you want to use.
  • Grab the Flex Tasks jar file from your SDK folder and place it into the libs directory of your project.
Here's the code we're going to be adding to the build file. This is a pretty simple file, and doesn't include many if any of the optimizations and tweaks that ANT provides.

We now need to get this running.


  • Open the ANT view in Flex Builder. You should have this already installed if you've followed the previous parts of the tutorial.
  • Choose to add a buildfile and choose the projects buildfile
We should see the below.
Double click the compile target, and assuming everything has been set up properly, the SWF will be generated in the target directory.
We can clean the folder structure back up by running the clean task, ready to be committed to source control once we are satisfied that the code works.

Part 1.2 Project Structure.

First of all create a new Flex LIBRARY project. Though we are going to be generating swfs, we aren't going to be having Flex Builder do it.

Why? Well we aren't going to be the only ones creating the swfs. In a properly ordered world, we developers only build our code to test it, make sure it's working, compiling, and then we leave it. If we want to run a Continuous Integration process for example, we need to be building the artifacts in some other way than FB.


Now create the folder structure the same as the image. I'll explain why we have all these top level folders.
  • conf - Short for configuration. This is where we are going to add the files that we use to dynamically configure the application at runtime. We keep these separate from the source files deliberately. These configuration files are able to be changed without (necessarily) recompiling the application.
  • libs - We keep all the libraries for the project. We will have a combination of Flex SWCs and Java JARs in here, split up by language.
  • setup - We are going to be keeping our build files in here along with any other files we use to generate artifacts. This might be additional config files for the Flex compiler for example.
  • src - Our main source directory. All our source code which will end up in production will end up in here.
  • target - This directory can be removed later on as we will be creating it on the fly. This is where all the compiled artifacts end up, this includes production SWFs, SWCs or Test SWFs.
  • test - Where all our test code is going. This includes mocks and additional files in order to create the unit tests.
  • util - This is where all the useful bits and pieces go that don't fit in elsewhere. Server start up scripts, Value Object generation scripts( like xDoclet ), etc.
In preparation for the next part of the tutorial, we're going to add some additional files.
  • Under source, add the following chain of folders com / codecradle / tutorial / application / view
  • In this final folder add a simple HelloWorld MXML application (Hint, don't choose new MXML Application under the context menu, choose MXML component and then set up the application manually. If you choose the MXML Application, FB tries to do all sorts of clever things we don't want it to do). We'll call this something nice and imaginative... like Tutorial.mxml ;)
  • Change the output folder in FB to "target" rather than "bin" just so FB doesn't keep trying to create it.
So by the end of this part, we should have something like the image below. Like all the other parts of this tutorial, add a comment if you have any questions!

More Flex : Hero information comes out

Deepa has furnished us with some more Hero related snippets, and they make some interesting reading.

Deepa's posting

The first spec of the Spark Datagrid makes an appearance, which as suspected, was closer to the Halo Datagrid rather than the AdvancedDataGrid. *makes sign to ward off evil*

There's some good reading in the rest of the spec's that have been posted up to the Open SDK site. The size report looks like it might be useful pre-production, and the RSL Enhancements is worth a read just to get more information on how the Flash Player handles loading of RSLs generally.

Other than this, we still need to wait to get a proper nightly build. Deepa's implication is that the impetus for this was on the Player team rather than Flex SDK team to get their stuff done (or maybe I'm reading too much into it ;) ) as Hero is closely tied to the new player features.

Lastly, it looks like Hero is semi-officially Flex 4.5 rather than 5. Rumours as to what will be in 5.0 starting in 3... 2... 1...

Wednesday, July 28, 2010

Stop using Maven for open source projects... please...

Honestly, just stop it.

Locking us into a particular build system is simply wrong. Say if a particular popular open source project uses Maven and I want to integrate it into a build system based off of ANT? No luck without a stupid amount of work.

Want to find the actual library a project is using so you can root around inside it? Good luck with that.

Compatibility issues between different versions means that I have to wipe my local repository when switching between two large projects (which I do frequently)... How is this saving me time again?

I have no problem bundling a build system with a project (it would be stupid not to) but Maven isn't just a build system, it's a dependency management system as well. If I was frothing at the mouth more, I'd call it a dependency obfuscation system...

Please, please, please... and I'm looking squarely at you Apache. If you must use Maven, include an alternative method of building the project for the rest of us.

Thursday, July 1, 2010

Bye Bye Slider, Hello Hero!

So Slider, the mobile version of the Flex framework is dead.

Instead, we are getting Hero, the code name for the new Flex framework. No word as yet whether this is a point release or Flex 5...

Deepa threw a whole load of links into her blog post about it. Most pertinent being...

http://opensource.adobe.com/wiki/display/flexsdk/Hero
http://labs.adobe.com/technologies/flex/mobile/faq.html

Most interesting part of this is finally a major update to the Datagrid! Anyone who's been kicking around from Flex will remember the last go at updating the Datagrid. The horrid AdvancedDataGrid which, and lets be fair to it here, was barely usable...

Once the news gets out in a proper way, I'd imagine the SDK team will be inundated with kitchen sink requests for a new Datagrid.

Which will we get from the Spark Datagrid... A decent replacement or ADG 2.0?

Monday, May 10, 2010

Part 1.1 Setting up the project.

Ok, so first things first we need to set up Flex Builder and ANT. There are many many blogs out there for doing this, so I'll not going to add to the noise. Instead, some handy links.
http://www.mobilefish.com/developer/flex/flex.html
http://www.roseindia.net/flex/flex_builder_ide.shtml
http://www.judahfrangipane.com/blog/?p=193
http://blog.jodybrewster.net/2008/04/09/installing-ant-in-flex-builder-3/

Go Ahead, get all that sorted out. Part 2 will be creating the project and how we are structuring it.

Friday, May 7, 2010

Added to Adobe feeds!

This is a good thing, and means I can actually start producing the tutorials so they will get indexed!

Tuesday, April 20, 2010

Learning Flex part 0 : Introduction

So here's what we are going to talk about over the next few posts. Each part of the tutorial will come with code snippets and assuming I keep it up, Flex Builder projects showing the code..

I'm also using Flex 3 rather than Flex 4. Rather than joining the cool kids showing off all the cool parts of Flex 4, we are sticking with the corporate standard. There are few organisations willing to take the leap to Flex 4 and Flash Player 10 yet. Until the penetration statistics rise, Flex 3 is where it's at.

So what's the project for this first tutorial? Very simple image viewer. We are going to start with a straight forward image, and add all sorts of cool stuff.

I may split some of the posts up if they get too big while I'm writing them, just so they aren't too overwhelming!
  • Part 1 : Setting up the project. Project structure. Setting up and using ANT. Using a Dependency Injection framework.
  • Part 2 : Constructing Objects. Always use interfaces. Compose objects as much as possible.
  • Part 3 : Testing the objects. Here's why we used interfaces. Using mocks for fun and profit. Using FlexUnit.
  • Part 4 : Composition over Inheritance. Adding a Zoom and Pan. Wrong way to do it with inheritance. Right way to do it with behaviours.
  • Part 5 : Adding additional controls. Zoom in / Out buttons.
This will be a good introduction to topics I'll revisit later, and provide a good grounding for adding in business logic and server interactions in later tutorials.

Learning Flex well rather than just learning.

There are plenty of blogs, tutorials, sites, books, people standing on soap boxes, and conferences that all want to teach you how to do Flex, but once you try to get beyond something simple, you run out of steam very quickly.

So the next few posts are going to be sequence of good practices which every good enterprise Flex developer needs to know. Soon, you'll be able to mix it with all those snooty Java developers who think their language is the only way to write enterprise apps!

Thursday, February 4, 2010

Checking analytics is working

Checking analytics is working

Monday, February 1, 2010

In the beginning... all was dark...

I thought about calling this blog "Yet Another Flex Blog", but my left brain talked my right brain out of it at the last minute.

Now that my blogging opportunities have opened up somewhat, I can finally start blogging the bits and peices that I've picked up over the last few years. Rather than splattering them all over Flex Coders whenever I get the chance.

Things I plan on covering over the first few months (chances are... years).

Setting up a java back end to use with Flex (could be re-titled, "Why not to use LCDS", I suppose)
Flex validators, why they suck, and how we can make them much better.
Novel ways to reduce your data centre's bandwidth using AIR 2.0 (Peer to Peer swarms aren't just for stealing music you know!).
Pushing workflows into all the tiers (ESBs are too heavyweight for client side development, lets come up with something a bit lighter, and less boring).
Frameworks, light is better yes?
Bits and pieces from my favourite Flex hobby, making a fully fledged generic forms and reporting application.