Alisha On Technology

Software Craftmanship Manifesto

2011-02-28T06:45:00.000-08:00

I want to pen down my thoughts on the Software Craftmanship Manifesto. So the first logical thing is to start by critiquing the manifesto and the wikipedia page on it.

"As aspiring Software Craftsmen we are raising the bar of professional software development by practicing it and helping others learn the craft."

There is quite a bit of background to this. Software Engineering is not a real profession with professional training, reliable skill assessment etc. Everyone calls himself a Software Engineer how do we really differntiate the really good ones from the average. Perhaps to reward the better ones or perhaps to create/promote more like the better ones.

Now unlike Law or Medicine there is a very little entry level barrier to software engineering, anyone and everyone who can write a script in an editor, compile it and get it to run is called an engineer. Step back: those are not the kind Google and Facebook hire do they? But it would be wrong to say Good Engineers work at Facebook and Google and the Not So Good Ones at the rest. Also this is unlike Medicine where there is a gradual progression with skill and experience acquisition.

Without a clear assessment of Software Skills it leads to hiring software engineers who belong to an industry vertical or know e.g. the capital markets. I would like to argue against that, but in a bit. I believe a Software Engineer should be assessed purely based on his Software Engineering Skills and not by the way of domain or vertical(s).

Do we need to raise the bar? Yes. But how? There is a growing concern of poor quality engineers at companies. To that I would say there are two kind of Software Professionals
1) Software Technicians
2) Software Engineers

The first ones are the ones who know the tools of the trade, and are the hands on the ground. They are associated with commodity work. There is no entry barrier to them. You refer to them at your companies by the word "Resources". The second category are Software Engineers, you usually know them by their 'name'. They are not resources.

I don't believe that the Software Profession needs a seperate Education or a cult system or a globally accepted master-apprenticeship model. While Software technicians come from diverse backgrounds some as less as a 5th grade education. An Engineer usually comes with a sound engineering (elec, mech, chem, computer science) or maths, physics education. The Foundations of Software Engineering are based on Maths, Data Structures and Algorithms. I find that engineers who have studied Donald Knuth's books to start with and have a strong knowledge of atleast one advanced engineering discipline as the most valuable engineers.

I'm not denying the Role of Experience in shaping an Engineer. I'm saying the differentiating factors for the best engineers have been laid way back in a quality and sound education. The rest are just Software Technicians. The biggest problems of the Software Industry are when Software Technicians become managers and preside over Software Engineers (My definition of those). The latter are very easy to identify and segregate.

I would like to state that its technicians who go in for certifications. The latter which may open doors but has poor corelation with the value one brings to the table. One of the reasons when I see one a many certifications I get suspicious.

"Not only working software, but also well-crafted software"
I would like to put the above this way. The best of the Engineers would work on Producing Valuable Software and in the process of doing that would be Making Valuable Contributions to the former objective. I would state this line as

NOT ONLY PRODUCING VALUABLE SOFTWARE BUT ALSO MAKING VALUABLE CONTRIBUTIONS TO IT

"Not only responding to change, but also steadily adding value"

SUPERCEDED BY ABOVE.

"Not only individuals and interactions, but also a community of professionals"

If the signatories of the manifesto is any indication. Then the community that is forming has no entry barrier, who are these people? How valuable are they as engineers? How will we ensure that worthy individuals have no barriers in being a part of the community and the unworthy ones are kept out? Will this be geography based with advantages to some locations or multiple communities across the Globe? I think the User Groups and Special Interest Groups have their place without a need for another cult of communities.

"Not only customer collaboration, but also productive partnerships"

Don't see how this impacts a craftmanship manifesto. Other than perhaps saying that Engineers who have productive partnerships and relationships are better and are worth more.

Dan North says "A really great programmer (and I’ve been lucky enough to work with a handful over the years) can out-perform a doing-it-for-the-money programmer by orders of literally hundreds, delivering in hours or days what would take an average developer weeks or months." in his article

In agreement I would tell Dan North you are comparing Engineers with Technicians.

Dan North goes on to say "How are those guys supposed to differentiate themselves? And how can they help others in the industry who have a real love and appreciation for the software they write? We need some sort of apprenticeship model, and a way to identify masters, both to apprentices and other masters. That sounds like the sort of model that craftsmen use."

How many apprentices will you take Dan? Or another Master take who is working a full time job. And if the Master is in Tokyo do we like in Medieval Ages expect the Apprentice to relocate to Tokyo bag pack and parcel? Unlike the Midieval European Masters who were independant practitioners/small-businessmen and could pay their apprentices, if I were a Master do I hire an Apprentice into my Employer's Organization? Isn't that empire building? And then will my master fire me and my apprentices together when we mess up or can they stay while I have to leave? Also do you think this will lead to I have more Apprentices than you thing? Looking at the list of signers on the manifesto?

In Essence I would conclude that the Craftmanship manifesto has no relevance/implications to any software organization or professional. There are better ways of dealing with excellance and medicrity in software than calling software a craft like music or dance.

All About Web Services:Soap, Rest, Hessian

2009-11-21T22:27:00.001-08:00

I have not included WCF Binary Protocols in the title because they are specific to the .Net/Windows platform. While creating interoperable services between multiple technologies we have to decide what to use.

I use SOAP for applications where Orchestration is required or which are more of business operations. SOAP provides the following which REST doesn’t.

WS-Addressing
WS-Policy
WS-ReliableMessaging
WS-SecureConversation
WS-Security
WS-SecurityPolicy
WS-Trust
WS-AtomicTransaction
WS-BPEL

I use REST primarily for Data CRUD Services. I usually need to send out a document along with the rest service url to the team which outlines the various operations that can be performed using it. With SOAP I just need to point them to the WSDL assuming they already know the domain/business functionality. If you are doing POSTS in REST then you are doing something wrong and need to move to SOAP.

While working with a UK Financial Client we came across a situation where the tree depth of the data structures was of the order of 20 and the payload was to the order of megabytes. While there are bigger issues but those apart we needed a binary protocol like Hessian which would work with C# and Java both. Like REST, Hessian also doesn’t have a service descriptor and we have to create a common XML file based on which we generate both the Java and .Net interfaces/data structures.

Being an Agilist I always have unit and integration tests around my services so I can maintain sanity in a complex environment with more than a 100 services.

Agile, Lean and Processes

2009-11-19T06:40:00.000-08:00

Firstly I agree with Agilists on lean processes. But I also agree with Carnegie Mellon Institute that Heavy Weight processes are required for Low Skilled, Secure environments like BPO's handling financial information.

While I think Agile works great for most of engineering. I think I will apply Lean to the rest of the company. I prefer Lean over Six Sigma primarily because Six Sigma is concentrated around a small band of Green and Black Belts while Lean involves the whole company.

The only problem I'm trying to avoid with Lean is that of Local Optimizations versus company and eco-system wide value chain optimizations. Any thoughts? email me!

Enterprise Tools... White/Black Lists and more

2009-11-19T06:34:00.000-08:00

I have worked in companies with very strict environments. And some where everything is open and upto me.

After looking at various companies from BPO's to Financial Institutions on one end and Agile companies on the other.

I think there is a need for
1) Application/Tool Whitelists
2) Application/Tool BlackLists
3) Recommended Applications
4) Banned/Illegal Applications/Tools
5) Standard Desktop and Server images for dynamic virtualized provisioning

From Legal & Security perspectives. While I expect employees to decide for themselves I also feel there should be a quick decision making for cases where executive decisions are required.

The question I'm trying to answer is how much time is good enough? 4 days? 2 days? or pick up the phone and talk?

SpringSource OSGi goes Agile June/09

2009-11-19T05:59:00.000-08:00

I was trying to integrate OSGi unit tests into the CI and came across the OSGi Test Stubs 1.0.0M1 release I guess as a die hard Agilist I have no complaints now.

Service Oriented Architecture (SOA) And My Take On It

2009-11-19T02:06:00.001-08:00

Who drives SOA?

The short answer is the Business/Consumer. Traditionally in enterprises we have had Sub-systems being developed to expose all services they can and then its upto the consumer to see how they want to use the services of the system. What I believe in is that the business should ask for what functionality they need and depending on that we should go down the SOA ecosystem to develop orchestrations, messaging and services to fulfill the business needs. As a rule every high level system should tell its service providers what they need to get their work done. This should flow down the tiers.

Service Sandbox & Testing

Every service should absolutely have two modes of operations, production and sandbox. Between them everything should be identical but the only difference should be that production mode works on production systems while the sandbox mode would switch transparently to a sandbox system which is a production identical system.

We need to test at two places to maintain sanity. The service provider should have CI integrated unit tests for each of the services they are exposing. And every consumer of a service should have integration tests to check if they are getting what they want from the services. Perhaps some Helper Services that only work in SandBox mode to aide Integration tests on the Consumers side, typically to insert test data.

Managed Extensions Framework/Unity for .Net

OSGi is java centric and there is almost no .Net equivalent for OSGi Java containers. MEF and Unity from Microsoft can resolve contract dependencies and are good at one consumer to many providers but lack versioning of contracts which we have to build ourselves. Another option is A .NET OSGi debutant.

OSGi Equinox and SpringSource DM Server

Eclipse contains an OSGi container called Equinox and if you see the evolution of Eclipse IDE’s then it is very clear that it is an Intra & Inter Enterprise SOA system. Typically in a SOA BootCamp I create 3 teams which create OSGi bundles and evolve them over versions and all the bundles are deployed into a central SpringSource DM Server. And I actually believe that the best way today to do SOA is to implement every system (Java) on DM Server with one module for every service provided and one for each service consumed.

Service Change Notifications

SOA Governance??? I doubt too much of it. I would rather have a central Service Change notification system rather than central SOA Governance, coupled with the business in the driving seat we have the equivalent of how the internet evolves. Loosely coupled, optimistic functionality (SOA Governance is pessimistic algorithmically)

So everytime a Service changes a notification is sent out and multiple versions of the service are provided, with a time to transition boundary. The notification should detail the change. Things like Service Signature(s) changing, input/output format/structure change and extra output content returned will need very little time to upgrade compared to the sole condition where extra input is required.

About Service Contracts Coarse/Fine grained and Chatty

This is what the conventional SOA discussions talk about and end after. Contrary to industry veterans I would rather have a Fine grained services by which I mean my service contract operation’s signatures can be fine grained, by coarse grained I would refer to an operation something like String Execute(String) where a huge XML is passed as string. I don’t actually like coarse grained service signatures but I like coarse functionality services. Lets not confuse between the two.

At this point I would like to step back and mention that I agree with the Industry when it comes to avoiding Chatty services. And with the above narrative SOA just boils down to each system exposing multiple versions of Services (They have to evolve their system accordingly).

By the way the SOA Manifesto has been signed it looks exactly like the Agile Manifesto. ;-) Cheers & Have a good one!!!

Parallel Programming & Concurrency Update

2009-07-13T08:00:00.001-07:00

Dynamic Non-Blocking ABP Work Stealing (http://research.sun.com/techrep/2005/abstract-144.html) is becoming the multiprocessor load balancing choice both in the academia and the industry. Moore’s law has stagnated in the sense that the processor clockspeeds are not increasing anywhere above 2.5-3 MHz now. The only way to scale is through multi core processors. This leads us to the challenge of utilizing the computing power available to us. And since the kind of programming languages we are used to are declarative i.e. we tell what needs to be done. Compilers simply do as we specify. There are no magic switches that we can toggle and parallelize our programs.

Proactive (Inria http://proactive.inria.fr/) did one framework for multicore distributed programming for Java which includes fault tolerance and load balancing using POJO’s. This is a C/MPI alternative with support for master-worker, divide and conquer, farm, pipe. If you are using Java and need High Performance Computing this is it.

Enter Microsoft with ParallelFX being released in .NET 4 with Task Parallel Library and Parallel Linq. Things like Parallel.For, Parallel.ForEach, Parallel.Invoke, .AsParallel() are common place with the bulk of infrastructure parallelization code being hidden from the developer. PLinq supports classic data parallelism.

Personally I think that implicit parallelism is much more difficult to declarative languages, for now I’m happy with the explicit parallelism available. I’m yet to explore F# to see what it offers in terms of Async Workflows using event thread combos. Till then adieu.