Altering the role of software architects to reduce code

The primary job function of software architects must be to reduce the amount of code being written. You could extend it to reduce internal communication, reduce dependencies, reduce complexity, and even reduce the number of components.

This way of defining an architect's role makes their engagement more relevant throughout the development process. Every code review now is an opportunity to understand the patterns and abstract them out for the developers. Every performance issue, code smells, and technical debt is an opportunity to take things out and eliminate the problems rather than fix them.

When developing software systems, the complexity and quality typically are directly proportional to the amount of code being written. And this is so independent of the experience and skill level of the developers. A good architect with an eye for patterns and an understanding of the business needs can iteratively make a big difference to what gets built.

Often I see software architect's doing the exact opposite of this, especially with the trend around micro-services over the last decade. The result is an architectural bloat consisting of technologies, components, and complexity.

Unfortunately, we see the magnitude of a complex system while overlooking the effort involved in building something simple. The result is a notion of the complex being harder and better than the simple. In effect, complex thus is also easier to build.

Finally, when you see the role of a software architect this way, they become invaluable to any development project. They are the innovators who find ways of doing more with less. Making systems robust by keeping them simple as they evolve.

Chicken & Egg of Entrepreneurship

Bootstrapping a business in some ways seems more like solving chicken and egg problems.

It is harder to hire since you don’t have a team. Harder to build a team since it is hard to hire.

Harder to get customers since you don’t have customers. You don’t have customers since it is harder to get customers.

You don’t have credibility since you don’t have enough customers. You don’t have enough customers since you don’t have credibility.

It is harder to raise money since you don’t have money. You don’t have money since it is harder to raise money.

Most problems in building a business boils down to chicken and egg problems. Once you are past that, you either have the chicken or the egg, the general economics and theories start to apply. Before that you have to solve the problem. I guess that is the main challenge of bootstrapping a business. That is the hard part. And maybe that is why it gets easier after one success.

There are multitudes of ways on how to get past them in every scenario. They have costs associated with them. Costs in terms of risk, investment, effort and time.

But modeling the bootstrapping process in this way simplifies them in a way. It is not that the world is coming against you, you simply are solving for something else. The perspective helps you see the problem in a different light. And thus helps you to look forward past it.

Modeling Knowledge, Understanding, Expertise, and Innovation

Why model Knowledge, Understanding, Expertise, and Innovation?

Modeling is a logical simplification of something that in reality could be quite complex, intangible, or abstract. A model then allows us to logically work with these concepts and influence them in a desirable way.

When it comes to the logical progression from Knowledge to Understanding to Expertise, understanding them in a way we can influence that progression is probably the most important thing we could learn. Without a simplified model, these concepts are hard to grasp and can take decades to understand.

This can also be very useful when building and managing large organizations especially in this hi-tech age of information and knowledge workers. A good model can guide how systems and processes are built so that organizations can drive towards innovation and change.

Knowledge-Understanding (KU) Graph

I have come to think of this learning progression as moving from a 1-dimensional space to higher dimensions. To start with, let us consider Knowledge as something that can be described on a 1-D space as points on a line of infinite knowledge. Thus, I could plot the Knowledge of someone on a line as follow -

Here each point on the line corresponds to some knowledge of the world and highlighted points are the Knowledge the person possesses. Knowledge is a complex enough construct to express in a binary term, but for our modeling, we simplify it as such.

There are practical applications of thinking about Knowledge in this way. We see its use all the time when hiring people when we specify a list of required skills. What we are really asking here is - do you have a Knowledge intersection with these set of Knowledge (Skills)?

Thinking along those lines, we can plot Understanding on the y-axis of the above graph. We frequently refer to the phrase "depth of understanding", and literally plotting it on the y-axis gives us a graphical representation of that phrase. The following diagram illustrates the Understanding of the Knowledge that the above person possesses on a y-axis (2-dimensional space).

I will refer to the above graph as KU graph here. Thinking of Understanding in this way makes understanding Understanding much simpler. The answer to the question "How well do you understand?" can now actually be plotted on a KU graph.

It also nicely separates out the fact that acquiring knowledge is not the same as understanding. Knowing is not understanding. It is far easier to Know. For example, you can read a book and probably know all about camping. It does not mean you understand camping or that experience.

Another example is that of compounding. You might think that if one knows how to add and multiply, they understand compounding, which is so not true. Someone, on the other hand, might not know how to add/multiply or could be really bad at it and they may still understand compounding well and would reap the benefits of that.

In life, we learn Knowledge at school. But using that Knowledge is what builds understanding. And that Understanding of the Knowledge is what comes in handy.

This model though might not be accurate. We have seen examples of prodigies who have been able to do exceptionally well in some field without ever having formally acquired the Knowledge. May be Understanding is not directly related to Knowledge and one can have a lot more Understanding with very little Knowledge. Or maybe Understanding can be acquired faster, almost instantaneously. We will not delve in those edge cases here.

Expertise

What makes one an expert in something? Can we extend this model to illustrate expertise?

Expertise is a relative term. It is relative to other people. So if someone says they are an expert, they simply mean they can do better than others for that skill.

Thus if we extend the KU graph model into the 3rd dimension, where the z-axis is the KU graphs of other people, we could visualize Expertise graphically via our model and therefore possibly influence it.

Modeling Expertise as a relative construct has other implications that we can see are true in real life. For example, for the same skills and understanding levels, one's Expertise will reduce over time as more people gain a similar understanding in that skillset.

Plotting this 3rd dimension on z-axis allows us to define levels of Expertise and calibrate it over time as KU graphs for a given Knowledge of the masses change. We have typically seen applications of this as skill rating, eg. a 5-star rating or a novice-intermediate-expert classification. The following slice of KU graphs of a set of people for a given skillset depicts this classification.

Thinking of Expertise in this way helps us to plan our way to expertise. It requires us to not only reach those depths of understanding but also reach those depths relative to others in that field.

We can see the analogies of this model in our education system. We do MS in say, Computer Science, after having studied computers for 4 years during undergrad, only so we can improve our understanding in that field. Nevertheless, that does not make us experts. We need to spend years in the industry working on a subset of that Knowledge set to gain understanding to an expert level. Alternatively, we can pick that subset and spend years doing a Ph.D. to reach the levels of being an expert.

Innovation

Now that we have modeled Knowledge, Understanding, and Expertise, where does Innovation happen? How does that spark of genius ignite? Or simply, how can someone be an innovative person?

If we think of the Expertise scale as ever-changing and KU graphs of the masses evolving for some K, my belief is that Innovation happens at the farthest depths of Understanding. 

One could graphically plot that region on our Expertise KU graph as follows -

Thinking of Innovation in this way can be very powerful since we can direct our way towards Innovation. Not only as individuals but as large organizations in general. We can also determine in what fields we want to be Innovative and how we can reach there.

This concept is not new. We see large amounts of funding directed towards a certain cause with a goal of improving innovation in those areas. Universities have been the most benefactor of this primarily because they organize Knowledge fields wells and thus make it easier to fund the cause. Enterprises dedicate research funds so they can invest in improving understanding in areas where they want to Innovate.

Finally

From my two decades of working and leading in various organizations, I have come to realize that having a model where we can drive towards Expertise and Innovation can be very useful. There are systems in place that help with them and having a simple underlying model can prevent them to degrade over time. Also, this path calls for tons of improvisation and calibration and such a model helps everyone understand the direction.