Context: Fast-Moving Global Organizations
IN THIS SECTION, YOU WILL: Understand the context in which the ideas in this book developed.
KEY POINTS:
- To better understand any idea or solution, it is crucial to understand the context in which this idea developed.
- The Grounded Architecture approach has evolved in the context of global, loosely coupled organizations that are diverse, with nonlinear growth dynamics, and under transformation pressures.
My approach to creating and running an architecture function is not an abstract idea. Instead, it is the generalization of lessons learned while solving specific problems in a particular context.
To truly grasp any idea or solution, you need to understand the context in which these ideas were born. I base my views on my time as a Chief Architect at AVIV Group and as a Principal Architect at eBay Classifieds and Adevinta. Let’s dive into some key characteristics of the organizational context that shaped my Grounded Architecture approach:
- Global scale: operating across multiple countries and continents with millions of users.
- Multi-dimensional diversity: the organizations I worked in were diverse, including customer base, workforce, business models, team topologies, and technology stacks.
- Nonlinear growth dynamics: besides organic growth, big organizations change their portfolio through mergers and acquisitions of new businesses or divestments.
- Synergies and transformation pressures: big organizations do not want just to be big. They want to exploit the benefits of the economies of scale and reduce duplication of efforts.
- Decentralized, loosely-coupled organizational units: organizational units have significant autonomy.
Global Scale
I have honed my approach within genuinely global and multicultural organizations on a massive scale:
- Operating across numerous geographies, cultures, and languages,
- Serving millions of users daily,
- Collaborating with thousands of software developers across hundreds of product and development teams,
- Implementing systems comprising hundreds of millions of lines of source code.
Operating on a global scale introduces several compelling opportunities for organizations. It can significantly increase organizational effectiveness by reducing duplication of effort through centralized shared activities. Additionally, leveraging economies of scale allows for cost advantages, such as lowering the unit prices of utilized technologies. Global operations also enhance business resilience and flexibility, enabling compensation for local market fluctuations with global resources. The expansive talent pool available to global organizations supports local and international initiatives. Moreover, these organizations possess significant resources to invest in supporting nonlinear growth through mergers and acquisitions (M&As).
However, the global and massive scale also presents numerous challenges. It results in high organizational complexity, with thousands of potential communication channels within the organization. The complex technology landscape entails numerous interconnected services. Managing a large talent pool incurs high workforce costs. Furthermore, such organizations face high computing resource expenses due to the need to serve a vast customer base around the clock. The operational complexity increases with high and variable customer demands across multiple locations. Additionally, global organizations have a vast attack surface, with many potential entry points for attackers. Lastly, any manual process, such as creating an organizational or technology landscape overview, is limited due to the scale involved.
Balancing these opportunities and challenges on a global scale has been one of the most challenging and rewarding aspects of my architectural work.
Multi-Dimensional Diversity
The organizations I worked with were incredibly diverse across multiple dimensions:
- Cultures: Collaborated with a varied workforce and clientele, both local and remote.
- Organization: Engaged with units of different sizes, complexities, and organizational styles.
- Product: Managed diverse features catering to various markets and customer segments.
- IT Architecture: Balanced legacy systems with modern approaches.
- Technology: Utilized numerous programming languages and thousands of third-party libraries, frameworks, and services.
I worked with units differing in several aspects:
- Unit Size: Ranged from hundreds of employees to just a dozen.
- Team Topologies: Spanned from single-team setups to hierarchical team organizations.
- Architectural Roles: Varied from having dedicated local architecture teams and lead architects to smaller units where team members handled architectural duties alongside other responsibilities.
We managed a range of styles in active production systems, from legacy monolithic applications to intricate modern microservice and serverless ecosystems. Each organizational segment had its own unique history and legacy systems.
Our technology stack was extensive, covering multiple mainstream technologies. The infrastructure included several public cloud providers (AWS, GCP, Azure) and custom-built private data centers. Our systems employed a variety of application technologies, such as:
- Database Technologies: MySQL, PostgreSQL, MongoDB, Cassandra, AWS RDS, and more.
- Backend Programming Languages: Java, C#, Go, Scala, PHP, Node.js, Kotlin, among others.
- Mobile App Programming Languages: Swift, Objective-C, Java, Kotlin, Flutter/Dart, and more.
- Frontend Programming Languages and Frameworks: React, AngularJS, Vue, jQuery, and others.
Diversity offers several opportunities, including:
- Increased Technology Innovation: A diverse workforce can explore more technologies and tools creatively.
- Better Implementation: Access to a broader pool of resources allows for selecting the best tool for the job.
However, diversity also brings challenges:
- Increased Complexity: Higher system landscape complexity and cognitive load for teams mastering numerous topics simultaneously.
- Reduced Flexibility: Expertise is spread across many domains and technologies, limiting reorganization possibilities.
- Higher Technical Debt: Multiple technology stacks can lead to increased legacy components and outdated technologies.
In conclusion, while diversity is a rich source of new possibilities from an architectural perspective, it also necessitates managing complexity carefully.
Nonlinear Growth Dynamics
Complex organizations like the ones I have worked in are often highly dynamic. These organizations frequently undergo significant growth, contraction, and reorganization, evolving both organically and inorganically.
Organic growth refers to internal expansion driven by the company’s own operations. Inorganic change involves acquiring other businesses, opening new locations, or divesting parts of the company.
Nonlinear growth, in particular, can be advantageous in several scenarios. It can rapidly increase the customer base or introduce new market segments. Additionally, such changes can accelerate innovation by incorporating new technologies or services.
However, nonlinear growth dynamics significantly impact architectural activities. The sudden integration of new companies increases organizational complexity, introducing many new units. Acquiring a new company also brings in new technology and engineering units, along with their unique processes and technology stacks. Furthermore, these nonlinear dynamics necessitate a flexible architecture to accommodate potential divestitures.
In summary, while nonlinear growth offers substantial benefits, it also presents challenges in managing increased complexity and maintaining architectural flexibility.
Synergy and Transformation Pressures
Complex organizations aim to grow not just in size, but also in efficiency by leveraging economies of scale, cost synergies, and enhancing their capacity for innovation. Our investors expect us to become more than the sum of our original parts.
Pursuing synergies and transformations offers several opportunities:
- Cost Reductions: Synergies lead to less duplication and lower expenses.
- Accelerated Innovation: Savings from cost reductions free up resources for innovation.
- Reuse and Sharing: Creating synergistic components enables more possibilities for reuse.
- Increased Efficiency: Well-executed transformations result in greater efficiencies and lower unit costs.
However, striving for synergies and efficiency presents challenges:
- Up-Front Investment: Significant initial investment is required to realize benefits, which carries high risks.
- Performance Pressure: Teams must deliver excellent short-term results while undergoing significant transformations.
- Temporary Productivity Drops: Balancing transformation activities with regular work can temporarily reduce productivity.
- Increased Complexity: Post-transformation, the organization and technology landscape may become more complex due to increased dependencies, such as reusing central services.
The pressure to achieve synergies and efficiency can lead to high expectations and complicate regular architectural work. Nonetheless, these forces also create numerous opportunities for growth and improvement.
Decentralization and Loose Coupling
Researcher Karl Weick developed the concepts of tight and loose coupling to describe organizational structures, initially in educational institutions and later applied to diverse businesses. According to Weick, a tightly coupled organization has mutually understood rules enforced by inspection and feedback systems. In such organizations, management can directly coordinate different departments’ activities according to a central strategy.
In contrast, a loosely coupled organization lacks some elements of a tightly coupled one. Employees have more autonomy, and different departments may operate with little coordination.
Due to historical and strategic reasons, most organizational units I worked with were loosely coupled. Our companies frequently grew through acquisitions of companies in different marketplaces. Business strategies also promoted the independent evolution of local units to address local market needs more effectively and quickly. These units often enjoyed a high level of autonomy, frequently with their development teams and sometimes with local CFOs, CMOs, or CEOs.
Loose coupling offers several advantages:
- Higher Flexibility: Units can develop independently, addressing specific needs without synchronizing with other units.
- Faster Time-to-Market: Fewer dependencies enable marketplaces to rapidly change and evolve their products for local needs.
- Innovation: Opportunities to quickly explore ideas in smaller contexts.
However, loose coupling also presents several challenges:
- Duplication of Effort: While local market needs differ, there is often significant overlap in product features and technology, leading to redundant efforts as each marketplace creates solutions for the same problems.
- Increased Accidental Diversity: Limited synchronization may result in significantly different design and technology choices for the same problem, making it challenging to consolidate solutions, move personnel between teams, or benefit from economies of scale.
- Limited Central Control: Fewer dependencies and varying goals make it more difficult to implement changes across the organization.
From an architectural perspective, loose coupling presents an interesting challenge as it often leads to a conflict between global alignment and control and local autonomy.
Questions to Consider
To better understand any idea or solution, it is crucial to understand the context in which these ideas developed. When using ideas from this book, ask yourself how your organizational context differs from mine:
- What are the unique characteristics of your organizational context?
- What is the scale of your organization? How it affects architecture function?
- How diverse is your organization?
- What are the growth dynamics of your organization?
- Are you experiencing synergy and transformation pressures?
- How (de)centralized is your organization?