Difficult to see. Always in motion is the future.
Master Yoda, Star Wars Episode V – The Empire Strikes Back (1980)
This article is purely my prediction on how things are going to be in next 8 years. I can be either 100% correct or 100% wrong – we are going to judge that in 2030. In this article I predict concepts and structure of Business-oriented Platforms for Composable Applications, settlement and standardisation of DevOps & Data Toolsets, Software Uberization and Architecture Trends. If you have a different opinion on what we, IT Engineers, will experience in incoming years – let me know!
When I have started my professional career in Software Engineering (2012/2013), the common practice was to build large systems packed with multiple functionalities, serving features to many different business domains. I was working in Integration Team, which role was to deliver a SOA-oriented Platform – so those large systems can sometimes talk to each other. We were trying to build a Canonical Data Models, which in theory were about to be understandable by each business unit. What appeared in practice – quite the opposite happened. We were releasing our software once a year, when each functionality had been finished – because our software was unable to be released in small, independent packages. Each release was a disaster challenge back then. Projects were long-lasting and costed millions of dollars. SAP (and large products like this) were kings, forcing organisations to change themselves in order to benefit from using expensive software products.
To be honest, when I’ve started working in such environment, I knew that this is something fundamentally wrong with such approach. But I couldn’t imagine how can it be organised differently – until I’ve heard about DevOps movement. It quickly became more and more popular – and I had a first chance to experience this different approach somewhere between 2016 and 2017. The Market already developed good practices around building more distributed software, using microservices patterns, cloud and emerging DevOps tools, so I was able to use all of it commercially in my software delivery initiatives. I have experienced a first revolution in how software can be delivered – as an architect, then a manager – and now as an advisor.
My gut tells me that this was not the only revolution I am going to see. We hear more and more about many promises Agile methodologies failed to deliver, problems with DevOps toolsets and lack of experts, collapsing economy and decentralisation of global world which will end up with looking for real reusability to still produce profit – and in all of that a new market opportunities on the horizon which Web 3.0 will soon bring us to the table. So today I will try to predict the future. This article is my guess how IT Industry will look like in 2030 – and how we are going to get there.
Agile failed…?
A nice clickbait, right? It’s very unpopular sentence to say – you are going to be eaten alive by Agile Gurus on LinkedIn. And they are probably going to be right. So let’s rephrase this sentence into something closer to the reality.
Agile failed to be properly understood and deployed in corrupt manner by many organisations. I see some similarities with socialism implementation by Soviet Union – in theory they were trying to change poor people’s lives empowering them with rights to production outcomes. In reality they have starved most of them and poisoned the mindset of those who survived for decades (visit Eastern Europe and talk to people outside big cities – you will understand). Putting aside the discussion if socialism is even deployable at all, it was definitely not deployable in 1917’s state of Russian Empire. Similar case is with Agile – it brought benefits to the teams with culture ready to embrace it. And brought chaos for others. Will those who failed try it again? Probably not. Will all organisations which haven’t adopted Agile Approach fail? Not likely as well.
You see… I think that Agile is not a single remedy for 100% of Software Delivery. You can still be quite ok without being fully flexible, without deliver software to production every day, without being self-organising within the Delivery Team. The problem I see with old-fashioned waterfall was not all that – we just had too long iterations with too big scopes. And then we tried to deliver everything with Scrum in particular, no matter if it was actually needed or organisation was ready for such flexibility.
So… what will happen in near future? I predict some renaissance of waterfall deliveries (named differently than Waterfall), but much better organised than it was before. Agile approach, even if presumed a failure in organisations, exposed a potential of decomposition and power of smaller functionalities released to be verified quickly. This will be adopted by this reborn waterfall – we won’t come back to big projects.
Don’t get me wrong – the flexibility of self-organising team can only be achieved in Agile approach – but it is not required every time. I think that being able to speak that out loud will begin a more smart approach to waterfall methodology – with more frequent releases, specialisation in business domains by delivery teams, close cooperation (or taking over) with Operations and Delivery Leaders instead of Project Managers – people understanding both technology and business value, and focused on the business outcomes more than just project scope.
Two Types of Software Delivery Teams
So am I claiming that Agile is dead and won’t be continued? Definitely not. Quite the opposite in fact – the ones doing Agile will do Agile, and the rest will simply stop lying and conclude iterative frameworks of fixed software delivery. My prediction is that in future admitting to not practicing Agile won’t be a shame like it is today.
What I notice today is that a lot of teams are agile just on paper. For example, when I ask new comers on interviews if they have worked in Scrum: what I hear is “yes, we have 2 week sprints, but no Product Owner and we deliver to production every month when we are ready“. What they do now is lying to themselves that they are agile, while in fact they just deliver planned scope of software in iterations. I think in nearest future people will simply start lying that they do agile when this renaissance waterfall becames popular (any idea on the name which will pop up for this?). But those who really practiced and mastered agile approach will remain agile – and take over the innovation and most challenging initiatives. Simply speaking – those agile guys will be doing a real money in IT Industry. Here is how I think we are going to have two-speed IT – and this is my vision on how it will look like.
If you are interested about the details – you can expand the table below.
Detailed comparison between those two types of teams: what will they deliver, how, what type of architecture they will produce – and explanation of my prediction.
| | Efficient Waterfall Teams | True Agile Teams |
|---|---|---|
| What will they deliver? | Composable Business Applications – they will build Business Applications upon Platforms which address not only technology topics (like DevOps Platforms) but also have ready-to-use distributed components for specific Business Domain. Their “ancestors” from old-fashioned waterfall were working on customisations – these guys will build something custom from ready to use components. A bit more about composable applications is here. | Innovative Products and Platforms – they will build unusual, game-changing tailor-made solutions and platforms upon which Efficient Waterfall Teams will then build customised solutions. They will be also working in R&D area. |
| How will they deliver? | Teams are going to be led by Delivery Leader – a person combining both technical and business skills (a real 50/50). I predict that in nearest future many Solution Architects will acquire business specialisation to play this role (for example: Solution Architect within Payments Domain in Banking). The role will be a combination of the architect and subject matter expert. Delivery Leader will manage the backlog by decomposing requirements for Business Domain into first business stories – and then deeper into technical tasks and architecture. This person will also assign and explain those tasks to the Team (so Developers won’t need to be that self-organising as in Agile Teams). The increment will appear on Production Environment within 3-months iterations. Responsibility will be handled mostly by this Delivery Leader. With this approach, Delivery Team will not be that close and self-organising as it is in Agile – which on the one hand makes this Delivery Leader a possible bottleneck – on the other, it will be just easier to combine experts worldwide, or even achieve an Uberization of Development Power (where one expert will be a part of many teams like that, composing the private backlog from many IT initiatives). | I hope we all know how a true agile works now 🙂 so in short: Teams will work either on Kanban (with a change-applicable list of stories to be delivered on Kanban board) or Scrum (with a business goal to be delivered every sprint). What will not change is that Teams will be self-organising and take responsibility for what they deliver. The scope of delivery will be selected based on market feedback – and we are going to have Product Owners making decisions and Delivery Teams executing them in best manner. Teams will also work as a one Unit – without shuffling team members. Team members will be much closer to each other than in Efficient Waterfall Approach. |
| Architecture Impact | Teams will use Platforms of Platforms building Composable Applications – which are going to be either separately-deployable services (microservices...but bigger?) or distributed monoliths. In general – those Teams are going to reuse a lot of staff the Agile Teams will produce. | Teams will deliver Platforms of Platforms – composable parts of Business Domain-oriented functionalities easy to use by the specific IT initiative. Teams will also deliver Tailor-made Microservice oriented applications. They are also going to deliver technology-oriented platforms to build software upon it (like DevOps, API Management, Data, Blockchain). |
| Expected market presence | My guess: 70% of IT Teams globally will work in that model. They will be divided on two types of Teams: focused more on Delivery and more on Operations, but unlike in the past – they will be working closely together by sharing team members or planning together. | My guess: 30% of IT Teams globally will work in that model. They will take responsibility for both Delivery and Operations, hiring Developers, Testers, SREs etc – people having full knowledge to deliver and operate. |
| Why do I predict that? | I think this is already happening – but many teams working like that is lying that they work in Agile. Not all organisations adopted well the flexibility and kind of uncertainty management which happens when we are truly agile. There are also many businesses and areas of businesses which do not require agile approach – and they just pretend to be agile in order not to scare engineers to work with them. And there is a trend now to be agile, so they don’t want to be untrendy. I also think that in order to be agile, you need to have a certain culture and mindset – which is not adoptable in each company or business unit. We have regulated markets on traditional financial or health sectors, which simply failed to acquire the necessary speed. Also the regulatory processes are stopping them to be agile. They also need a way to deliver software better than with old-fashioned waterfall. The last argument is… hiring options. My opinion is that many developers simply want to be assigned with properly-defined tasks and do their great job uninterrupted. Many are not the best team players. Many are working remotely which makes a scrum team harder to integrate. Those sentences are my observations, so I can be wrong – but as a consultant, I take part of multiple initiatives, and everywhere I see those trends. This 70% I predict comes from that – and also from slightly modified Pareto Rule. | Well – besides the organisations, domains or teams unable to be truly agile (because of many reasons) – I also experienced the ones which acquired agile as their natural environment. And those teams are achieving business goals in time and scope unimaginable for others. They will simply not abandon the approach and mindset that makes them successful. I said that 30% of teams in the future will be like that. Those teams will deliver more difficult topics than the rest – being more successful on software delivery. What I think as well is that members of such teams will just earn much more than members of efficient waterfall teams. This prediction is also mostly from my experience – where I see career (and income) growth much faster for developers with business-oriented mindset than the ones with technology-only focus. What I hope for is that I’m wrong – and the percentage of such teams will be bigger! |
Architecture & Technology Trends
Alright, enough of organising the work – let’s talk about the technicalities. I think we are going to have even bigger revolution here. Those changes I predict are a natural (for me) continuation of processes already happening. In short:
- we will finally stabilise DevOps Technologies which are a headache right now, we are going to have similar headache with blockchain;
- we will move into uberization of IT Services which will be visible in architecture oriented around platforms for rapid business software delivery;
- we are about to divorce crypto and Web 3.0 when business organisations will start utilising the second one.
Real Revolution: Business Domain-Driven Platforms for Composable Applications
I already mentioned something about business-oriented Platforms and overall business-domain specialisation within Delivery Teams. What I think is going to happen is appearance of Composable Business Platforms – where you are going to build your business processes out of predefined, easy to integrate, business-focused components – a bit bigger, but standardised microservices. This is what I claim to happen especially for big, traditional, repetitive businesses like banking, insurance, production – or repetitive domains like human resources, logistics, marketing. The difference between old-fashioned “out-of-the-box” products is that here only the components are going to be out of the box platform, not the whole solution. How will it work? Let’s talk about it on banking example.
How it is now: We have a Bank A. Within this Bank, we have Loans Domain, and we want to create a digital space for customers for getting Credit Cards online. What we do know is agree on how the process will look like – and normally we will end up with checking if the Customer is a real, validated person; then if we have any relation with the Customer within the Bank; then the scoring, finally – offer presentation, approval method plus a way the Customer will obtain the Card. It can differ between banks, but in general, process looks similar. So what architects will do within microservice architecture? They can create a service for Customer Lending, taking care of the process; a service for a scoring process; a service for provisioning the card to the customer – etc. And then we have a Bank B, with a bit different process… but in the end of the day, architects will create a bit different service for Lending, Scoring, Card Provisioning… repeating very similar work. This is how it looks right now, because we don’t have a Lending Platform yet (or do we…?). We have Lending Products – but they are close-coupling and are implementing full lending process. We don’t want that, because we want to be flexible within our business process, incorporating steps which are specific for Bank B.
How it can work in the future: We have Bank, and we have a Lending Platform. Platform is already providing components to verify customer, to track business process, to create an offer, to do logistics around the cards, for scoring – etc. What Bank A will do: design the full process, implement components for internal Customer checks (relation with the bank for example) – then customise (if necessary) the Platform components, and compose the whole solution. Having engineers specialised within this Business Domains & this particular Lending Platform – Delivery Lead will create tasks for developers around the world (uberization – see below) to make the components ready – and then put it together and release for the Business. Depending on Bank A or Bank B, custom components and overall process will be different – but they are going to incorporate business platform components within their solutions to speed up the delivery.
Of course, not all processes or businesses are going to be platformized. Like Uber – it is innovative, because they have created a platform for taxi drivers – something not existing (or not popular) on the market back then. There is also need for a creative approach for building such Platforms and changing them rapidly based on market feedback. Next thing is also a space for specific, unique processes and software ideas. Those initiatives I think will still be done by Agile Teams (because such initiatives are what they are the best for).
To conclude that part (and go deeper into software uberization) – in my opinion we are going to have two types of delivery teams – but engineers in both will need to specialise not only within the technology, but also within business domains. We already spent too much time with tech-oriented engineers to understand totally new business domain within every initiative. This cognitive load is one of the inefficient parts of today’s IT, which I think will be soon eliminated by business specialisation of software engineers.
Because of Platforms we are going to have Software Development Uberization & Low-Code Renaissance
What I predict with Composable Business-oriented Platforms will happen is a new type of software delivery – which I called Uberization of Software Development. How can it work? Well, we are going to have applications which will combine jira and uber functionalities, making business-specialised engineers within business-oriented Platforms to accept well-defined tasks to be done within their private backlogs. Each Task will have a detailed description on what is expected to be customised within composable part of Platform, with a price on it and acceptance criteria. This will also create a dedicated code branch within the Softbare (Uber for Software… I am not good with names), on which the Developer will contribute. Once it is finished, Delivery Leader will merge all of the results according to agreed process with business stakeholders. Developers don’t even need to know which company or initiative is creating the task for them – they won’t probably care. What they will see in their end is their own, private backlog, composed with tasks from many organizations – with a compensation assigned to the task by this organization. They can either accept it, or reject it – like Uber drivers does today with the rides.
What I predict as a next step after business platformization is a renaissance of Low-Code. Those Platforms, investing into standardisation of requirements gathering and specialisation of components will embrace the approach for building software components like lego blocks, with pre-defined elements. I think in time those business-oriented platforms will introduce low-codes specialised for particular businesses. This is what I think was missing in previous approaches to low-code, which failed because of specific requirements.
Of course such way of work will apply only for Engineers specialised within Business Platforms – where pre-defined components will have worldwide-known standards for tasks definition. It will not be applied to Agile Teams, who will be working on fully customised solution, being responsible for that – where the innovation comes from the whole Team.
Technology Trends: DevOps & Data Tools will finally settle.
Today you can play games like “is it Data/DevOps Tool – or a Pokemon?”. I am not joking, you can play that here. And you will win if you are really lucky. Funny, right? Well, it’s not for Enterprise Architects and Delivery Managers. It’s actually a disaster on which we are about to fix our architecture landscapes in near future.
Today, once a year or two we need to convince our stakeholders/IT-investors that we really need to change one CI/CD Tool to another CI/CD Tool, just because the current one is blocking our operating model and not allowing us to growth. Even if the tool is just 2 years old in the organisation, and was the best back then. This situation though is nothing different than what we had in the 90s – where Software Engineers had to choose from so many developing languages, communication patterns or information storage options. Luckily for us, at some point the market concluded and popularised the biggest ones. Today we mostly use Java and .Net for backends, Angular and React for frontends, HTTPs with REST/SOAP for communication, SQL is very similar no matter which Database you choose etc. The technology stack is much more stable, and we don’t spend so much time selecting technology for our business applications – like we spend this time with DevOps & Data Tools. What I think that this standardisation will finally be reached also with DevOps & Data Toolset. The market have a business case to do that – DevOps & Data Engineers are expensive and difficult to hire and mess done with countless technologies to enterprise architecture becomes a huge operational issue.
What I predict is Github and Elastic Stack will win this DevOps Toolset race for on-prem / cloud-agnostic approach (where Github will standardise the Dev part of application lifecycle, and Elastic will take over Ops). I am not sure yet which approach will win the Data Race – but within DevOps, I think in 2030 majority of Software will have Code, Artifacts & Security Credentials stored in Github-created toolset, with Github-predefined blocks to build CI/CD pipelines no matter where we deploy. Once the application lands on production environment: logging, tracing, monitoring and analytics will be done by Elastic Stack (also tailor-composed like lego blocks). And speaking of environment…
Cloud is finally becoming a King these days. They are going to finally produce an integrated DevOps & Data Platforms, easily composable or adjustable for operating model the company selects (I heard a rumor yesterday that Azure is already close to that approach). What I predict it will even growth bigger, taking over most of on-prem environments till 2030. But not all. Why? We still have regulatory markets which won’t change that simply – and I think that in order to really embrace benefits of the cloud, you need to change the way you think about IT – as an investment and business game changer, not a cost. I don’t believe that change in mindset will be played for each organisation till 2030. So I predict two approaches for environment management:
- For Cloud: I think serverless platforms will take over applications running now on container orchestration platforms, integrated with CI/CD served by Github and Ops served by either Elastic or cloud-native toolsets.
- For On-prem: I think two approaches are going to happen.
- First, we are going to have a federation of on-prems between organisations, creating a shared-cloud. Two or more organisations are going to have an agreement on common infrastructure, powered with some services. Here I think a serverless toolsets for running software will start to appear, working similarly to the ones in the cloud (but within limits of share for each organisation – and of course, resources limits).
- Second, we are still going to have container-orchestration toolset on smaller-scale on-prems – but simplified in terms of management (predefined and composable).
Conclusion
If you are interested in technology trends – Gartner is releasing their predictions every year. The idea of composable applications and Cloud Services composing DevOps Platforms which I mention in this article are already being mentioned there. They are much more advanced in their predictions that I am, providing some studies and research. The most fresh predictions from Gartner you can see here.
Initially I was also planning to write something about Web 3.0 and decentralisation and privatisation of Data, but… this article would be way to long (it already is). So maybe next one will be about that… But in short, if you are worried that you haven’t invested in Cloud Engineering Skills in 2010 to be a very expensive Cloud Architect with 12 years of experience today – you should invest in learning Blockchain in order no to miss being such expert in 2030. Web 3.0 and Blockchain approach to Data is another revolution which is coming.
In this article I also haven’t even touched Artificial Intelligence or Internet of Things (both growing rapidly). Those topics are not particularly in my expertise now (maybe it should?), so it is hard for me to claim with some level of certainty about the future of those areas. However, I will just publish two interviews with Bill Gates – who is much smarter than I am, and whom I believe with the predictions:
