Building Scalable Web Applications

Microservices won’t fix bad design—they’ll amplify it. Splitting a monolith into microservices doesn’t magically solve scaling issues. It introduces network latency, data consistency challenges, and operational complexity. Before going micro, ask: • Does each service have a clear, single responsibility? • How will services communicate—sync or async? • Can failures be isolated without breaking the whole system? • Have we properly defined bounded contexts? • Who will be the owner of that new service? • Is the team structured for microservices, or will this cause silos and slowdowns? • Can you afford it? Microservices work best when your architecture demands it, not when hype drives it. Choose wisely.

Mastering the API Ecosystem: Tools, Trends, and Best Practices The image I recently created  illustrates the diverse toolset available for API management. Let's break it down and add some context: 1. Data Modeling:    Tools like Swagger, RAML, and JsonSchema are crucial for designing clear, consistent API structures. In my experience, a well-defined API contract is the foundation of successful integrations. 2. API Management Solutions:    Platforms like Kong, Azure API Management, and AWS API Gateway offer robust features for API lifecycle management. These tools have saved my teams countless hours in handling security, rate limiting, and analytics. 3. Registry & Repository:    JFrog Artifactory and Nexus Repository are great for maintaining API artifacts. A centralized repository is key for version control and dependency management. 4. DevOps Tools:    GitLab, GitHub, Docker, and Kubernetes form the backbone of modern API development and deployment pipelines. Embracing these tools has dramatically improved our delivery speed and reliability. 5. Logging & Monitoring:    Solutions like ELK Stack, Splunk, Datadog, and Grafana provide crucial visibility into API performance and usage patterns. Real-time monitoring has often been our first line of defense against potential issues. 6. Identity & Security:    With tools like Keycloak, Auth0, and Azure AD, implementing robust authentication and authorization becomes manageable. In an era of increasing security threats, this layer cannot be overlooked. 7. Application Infrastructure:    Docker, Istio, and Nginx play vital roles in containerization, service mesh, and load balancing – essential components for scalable API architectures. Beyond the Tools: Best Practices While having the right tools is crucial, success in API management also depends on: 1. Design-First Approach: Start with a clear API design before diving into implementation. 2. Versioning Strategy: Implement a solid versioning system to manage changes without breaking existing integrations. 3. Developer Experience: Provide comprehensive documentation and sandbox environments for API consumers. 4. Performance Optimization: Regularly benchmark and optimize API performance. 5. Feedback Loop: Establish channels for API consumers to provide feedback and feature requests. Looking Ahead As we move forward, I see trends like GraphQL, serverless architectures, and AI-driven API analytics shaping the future of API management. Staying adaptable and continuously learning will be key to leveraging these advancements. What's Your Take? I'm curious to hear about your experiences. What challenges have you faced in API management? Are there any tools or practices you find indispensable?

Security can’t be an afterthought - it must be built into the fabric of a product at every stage: design, development, deployment, and operation. I came across an interesting read in The Information on the risks from enterprise AI adoption. How do we do this at Glean? Our platform combines native security features with open data governance - providing up-to-date insights on data activity, identity, and permissions, making external security tools even more effective. Some other key steps and considerations: • Adopt modern security principles: Embrace zero trust models, apply the principle of least privilege, and shift-left by integrating security early. • Access controls: Implement strict authentication and adjust permissions dynamically to ensure users see only what they’re authorized to access. • Logging and audit trails: Maintain detailed, application-specific logs for user activity and security events to ensure compliance and visibility. • Customizable controls: Provide admins with tools to exclude specific data, documents, or sources from exposure to AI systems and other services. Security shouldn’t be a patchwork of bolted-on solutions. It needs to be embedded into every layer of a product, ensuring organizations remain compliant, resilient, and equipped to navigate evolving threats and regulatory demands.

Netflix recently got a lot of hate during the Mike Tyson vs Jake Paul fight, leaving over 100,000 viewers frustrated as they couldn’t see about 50% of the match because of buffering. Critics were quick to compare this to Hotstar, which smoothly managed 59 million concurrent viewers during the 2019 Cricket World Cup. So, what made Hotstar’s infrastructure stand out? Here’s how they did it:   1️⃣ Predicting Traffic Spikes   Hotstar knew traffic wouldn’t just grow—it would explode during key moments like tosses, boundaries, or MS Dhoni’s entrance.  - 1 Million Users Per Minute: Hotstar saw a rapid influx at peak moments.   - 5 Million Stayed During Rain Delays: Their infrastructure held strong even when fans waited for hours.   - Lesson: Understand your audience’s behavior and prepare for the most demanding scenarios.   2️⃣ Custom Autoscaling   Traditional autoscaling wasn’t enough. Hotstar engineered their own tools to handle extreme demands:  - Proactive Scaling: Infrastructure was scaled up in advance to deal with traffic surges.   - Quick Downscaling: Resources were released as traffic dropped, ensuring cost efficiency.   - Lesson: Build tools that adapt to your platform’s unique needs, especially during live events.   3️⃣ Resilience Through Chaos Engineering   Hotstar stress-tested their systems relentlessly:  - Simulated 50 Million Concurrent Users: Distributed across eight AWS regions to avoid bottlenecks.   - Tested Failures: Simulated network outages and delays to ensure redundancy.   - Lesson: Push your system to its limits before real-world users do.   4️⃣ Focusing on Core Services   During critical moments, Hotstar prioritized what mattered most:  - Disabled Non-Essentials: Recommendations and personalized features were temporarily turned off to focus on streaming.   - Pre-Warmed Servers: Servers were prepared to handle massive spikes without delay.   - Lesson: Prioritize core functionality during high-stakes events, even if it means sacrificing extras.   5️⃣ Strategic Push Notifications   Hotstar used push notifications to re-engage millions of users:  - 150-200 Million Notifications Sent: These drove an additional 6 million viewers within minutes.   - Lesson: Notifications can drive engagement but require robust infrastructure to handle the surge.   The Outcome   Hotstar didn’t just manage—they thrived:   - 59 Million Concurrent Viewers: A global record.   - 10 TB/Second Bandwidth: Leveraging 70-75% of India’s internet capacity.   - Zero glitches during peak traffic moments.  Netflix’s outage shows how even the biggest platforms can falter under extreme demand. Hotstar’s 2019 Cricket World Cup playbook proves that with the right preparation, live events can be seamless even at record-breaking scales.  The question remains: Will Netflix take notes from Hotstar’s victory? 

You can’t design an efficient system without mastering these two core concepts: Throughput and Latency. Understanding the trade-offs between them is non-negotiable if you’re diving into system design. ♦ Throughput Throughput refers to how much data or how many requests a system can process in a given period. It’s typically measured in transactions per second (TPS), requests per second (RPS), or data units per second. Higher throughput means the system can handle more tasks in less time, making it ideal for high-demand applications. How to Increase Throughput: - Add more machines (horizontal scaling) - Use load balancing to distribute traffic evenly - Implement asynchronous processing with message queues ♦ Latency Latency is the time it takes for a system to process a single request from start to finish. It’s usually measured in milliseconds (ms) or microseconds (µs). Low latency is crucial for systems where quick responses are critical, such as high-frequency trading or real-time messaging. How to Reduce Latency: - Optimize code for faster execution - Use faster storage solutions (like SSDs or in-memory databases) - Perform database tuning to reduce query times - Implement caching to serve frequently used data quickly ♦ The Trade-off: Throughput vs. Latency These two metrics often pull in opposite directions. Increasing throughput might lead to higher latency, and reducing latency might limit throughput. For example: - Asynchronous processing boosts throughput by queuing tasks but can delay individual task completion. - Extensive caching reduces latency but requires more memory and careful management to prevent stale data. The key is balancing throughput and latency based on your system’s needs. A high-traffic e-commerce site may prioritize throughput, while a stock trading platform will focus more on minimizing latency. Understanding these trade-offs is essential for building scalable and responsive systems.

💬 Last November I had a call with the CEO of an emerging health platform. She sounded very concerned -- "Our growth's hit a wall. We've put so much into this site, but we're running out of money and time. A big makeover isn’t an option, we need smart, quick fixes." Looking at the numbers, I noticed: ✅ Strong interest during initial signups. ❌ Many users gave up after trying it just a few times. ❌ Users reported that the site was too complicated. ❌ Some of the key features weren’t getting used at all. Operating within the startup’s tight constraints of time and budget, we decided on the immediate plan of actions-- 👉 Prioritized impactful features: We spotlighted "the best parts". Pushed secondary features to the backdrop. 👉  Rethought onboarding: Incorporated principles from Fogg's behavioral model: • Highlighted immediate benefits and rewards of using the platform (motivation) • Simplified tasks, breaking down the onboarding into easy steps (ability) • Nudged users with timely prompts to explore key features right off the bat (triggers)    👉 Pushed for community-driven growth: With budget constraints in mind, we prioritized building an organic community hub. Real stories, shared challenges, and peer-to-peer support turned users into brand evangelists, driving word-of-mouth growth. 👉  Started treating feedback as "currency": In a tight budget scenario, user feedback was gold. An iterative approach was adopted where user suggestions were rapidly integrated, amplifying trust and making users feel an important part of the platform's journey. In a few months time, the transformation was evident. The startup, once fighting for user retention, now had a dedicated user base, championing its vision and propelling its growth! 🛠  In the startup world, it's not just about quick fixes, but finding the right ones. ↳ A good UXer can show where to look. #ux #startupux #designforbehaviorchange   

User experience surveys are often underestimated. Too many teams reduce them to a checkbox exercise - a few questions thrown in post-launch, a quick look at average scores, and then back to development. But that approach leaves immense value on the table. A UX survey is not just a feedback form; it’s a structured method for learning what users think, feel, and need at scale- a design artifact in its own right. Designing an effective UX survey starts with a deeper commitment to methodology. Every question must serve a specific purpose aligned with research and product objectives. This means writing questions with cognitive clarity and neutrality, minimizing effort while maximizing insight. Whether you’re measuring satisfaction, engagement, feature prioritization, or behavioral intent, the wording, order, and format of your questions matter. Even small design choices, like using semantic differential scales instead of Likert items, can significantly reduce bias and enhance the authenticity of user responses. When we ask users, "How satisfied are you with this feature?" we might assume we're getting a clear answer. But subtle framing, mode of delivery, and even time of day can skew responses. Research shows that midweek deployment, especially on Wednesdays and Thursdays, significantly boosts both response rate and data quality. In-app micro-surveys work best for contextual feedback after specific actions, while email campaigns are better for longer, reflective questions-if properly timed and personalized. Sampling and segmentation are not just statistical details-they’re strategy. Voluntary surveys often over-represent highly engaged users, so proactively reaching less vocal segments is crucial. Carefully designed incentive structures (that don't distort motivation) and multi-modal distribution (like combining in-product, email, and social channels) offer more balanced and complete data. Survey analysis should also go beyond averages. Tracking distributions over time, comparing segments, and integrating open-ended insights lets you uncover both patterns and outliers that drive deeper understanding. One-off surveys are helpful, but longitudinal tracking and transactional pulse surveys provide trend data that allows teams to act on real user sentiment changes over time. The richest insights emerge when we synthesize qualitative and quantitative data. An open comment field that surfaces friction points, layered with behavioral analytics and sentiment analysis, can highlight not just what users feel, but why. Done well, UX surveys are not a support function - they are core to user-centered design. They can help prioritize features, flag usability breakdowns, and measure engagement in a way that's scalable and repeatable. But this only works when we elevate surveys from a technical task to a strategic discipline.

🚀 Nothing kills a 10M TPS load test faster than load imbalance. Production spikes? Even tougher. Load balancing might not be glamorous, but it’s a crucial component in scaling fleets efficiently. 1️⃣ The Power of Vertical Scale 10M TPS are easy to handle on a 1000 webserver fleet right? Afterall, that's only 10K TPS per server, assuming a perfectly balanced load. But in practice, the busiest 1% of servers (10 nodes in 1000) often get 10x the load, destroying tail latencies, & causing load based outages. The antidote to the law of large numbers is to pursue smaller numbers. Momento routing fleet handles 200K+ TPS on a single 16-core host. This took significant investment considering we had to terminate TLS, authenticate, route to the storage shards, and collect deep metrics, while maintaining our p999 SLO. The effort pays off by minimizing statistical imbalance in larger fleets. 10M TPS only requires 50 hosts. 2️⃣ Load Balancing in Spikes Demands Load Shedding Your fleet hums along at 400K TPS with two routers across two AZs. Suddenly, a *1M TPS spike** hits. Autoscaling? Too slow— server side connections are already established and getting hotter. Even if we are proactive about scaling, we still need to redistribute load to new routers. Enter load shedding: with well behaved clients (this is why we write our own SDKs), you can send an HTTP2 GoAway frame and expect your clients to establish connections, rebalancing load to the newly provisioned capacity. This magic trick minimizes over provisioning, reduces waste, & keeps us within our p999 SLOs with a leaner fleet. 3️⃣ Hot Keys? Use the Force, Luke: Cache the Cache! Hot keys are a nightmare. Millions of reads on the same key can crush storage shards. Adding replicas? Too slow and inefficient. Adding replicas proactively requires over-provisioning your entire storage fleet. Doing it on-demand for the hottest shard takes too long. Momento routers simply sample the hottest keys, caching them locally. By caching a hot key for just 10ms, we reduce storage load from 1M TPS to 500 TPS across 5 routers (each router handling 200K+ requests). This is incredibly efficient for a 10ms trade-off on consistency. For customers needing more consistent reads (e.g., nonces), a consistent read concerns allows them to trade off hot key protection for data that is 10ms fresher. I have yet to run across a customer that needs millions of consistent reads on a single key. These simple techniques—plus multi-tenancy—help us deliver robust systems ready for production spikes of 10M+ TPS. 💡 Want to nerd out with me on your high-volume workflows? Book time with me for Scaling Office Hours: 👉 https://lnkd.in/g6fdXJWV 💬 If you think there are techniques we’re missing or should reconsider, please add them in the comments. We are eager to learn more and continue improving the robustness of our systems every day!

You dockerized your .NET Web apps. Great, but next you'll face these: - How to manage the lifecycle of your containers? - How to scale them? - How to make sure they are always available? - How to manage the networking between them? - How to make them available to the outside world? To deal with those, you need 𝗞𝘂𝗯𝗲𝗿𝗻𝗲𝘁𝗲𝘀, the container orchestration platform designed to manage your containers in the cloud. I started using Kubernetes about 6 years ago when I joined the ACR team at Microsoft, and never looked back. It's the one thing that put me ahead of my peers given the increasing move to Docker containers and cloud native development. Every single team I joined since then used Azure Kubernetes Service (AKS) because of the impressive things you can do with it like: - Quickly scale your app up and down as needed - Ensure your app is always available - Automatically distribute traffic between containers - Roll out updates and changes fast and with zero downtime - Ensure the resources on all boxes are used efficiently How to get started? Check out my step-by-step AKS guide for .NET developers here 👇 https://lnkd.in/gBPJT6wv Keep learning!

Data Products are NOT all code, infra, and biz data. Even from a PURE technical POV, a Data Product must also have the ability to capture HUMAN Feedback. The User’s insight is technically part of the product and defines 𝐭𝐡𝐞 𝐃𝐚𝐭𝐚 𝐏𝐫𝐨𝐝𝐮𝐜𝐭’𝐬 𝐟𝐢𝐧𝐚𝐥 𝐬𝐭𝐚𝐭𝐞 & shape. This implies Human Action is an integrated part of the Data Product, and it turns out 𝐚𝐜𝐭𝐢𝐨𝐧 𝐢𝐬 𝐭𝐡𝐞 𝐩𝐫𝐞𝐥𝐢𝐦𝐢𝐧𝐚𝐫𝐲 𝐛𝐮𝐢𝐥𝐝𝐢𝐧𝐠 𝐛𝐥𝐨𝐜𝐤 𝐨𝐟 𝐟𝐞𝐞𝐝𝐛𝐚𝐜𝐤. How the user interacts with the product influences how the product develops. But what is the 𝐛𝐫𝐢𝐝𝐠𝐞 𝐛/𝐰 𝐃𝐚𝐭𝐚 𝐏𝐫𝐨𝐝𝐮𝐜𝐭𝐬 𝐚𝐧𝐝 𝐇𝐮𝐦𝐚𝐧 𝐀𝐜𝐭𝐢𝐨𝐧𝐬? It’s a 𝐆𝐎𝐎𝐃 𝐔𝐬𝐞𝐫 𝐈𝐧𝐭𝐞𝐫𝐟𝐚𝐜𝐞 that doesn’t just offer a read-only experience like dashboards (no action or way to capture action), but enables the user to interact actively. This bridge is entirely a user-experience (UX) problem. With the goal of how to enhance the User's Experience that encourages action, the interface/bridge between Data Products and Human Action must address the following: 𝐇𝐨𝐰 𝐭𝐨 𝐟𝐢𝐧𝐝 𝐭𝐡𝐞 𝐫𝐢𝐠𝐡𝐭 𝐝𝐚𝐭𝐚 𝐩𝐫𝐨𝐝𝐮𝐜𝐭 𝐭𝐡𝐚𝐭 𝐬𝐞𝐫𝐯𝐞𝐬 𝐦𝐲 𝐧𝐞𝐞𝐝? A discovery problem addressed by UX features such as natural language search (contextual search), browsing, & product exploration features. 𝐇𝐨𝐰 𝐜𝐚𝐧 𝐈 𝐮𝐬𝐞 𝐭𝐡𝐞 𝐩𝐫𝐨𝐝𝐮𝐜𝐭? An accessibility problem addressed by UX features such as native integrability- interoperability with native stacks, policy granularity (and scalable management of granules), documentation, and lineage transparency. 𝐇𝐨𝐰 𝐜𝐚𝐧 𝐈 𝐮𝐬𝐞 𝐭𝐡𝐞 𝐩𝐫𝐨𝐝𝐮𝐜𝐭 𝐰𝐢𝐭𝐡 𝐜𝐨𝐧𝐟𝐢𝐝𝐞𝐧𝐜𝐞? A more deep-rooted accessibility problem. You can't use data you don't trust. Addressed by UX features such as quality/SLO overview & lineage (think contracts), downstream updates & request channels. Note that it's the data product that's enabling quality but the UI that's exposing trust features. 𝐇𝐨𝐰 𝐜𝐚𝐧 𝐈 𝐢𝐧𝐭𝐞𝐫𝐚𝐜𝐭 𝐰𝐢𝐭𝐡 𝐭𝐡𝐞 𝐩𝐫𝐨𝐝𝐮𝐜𝐭 & 𝐬𝐮𝐠𝐠𝐞𝐬𝐭 𝐧𝐞𝐰 𝐫𝐞𝐪𝐮𝐢𝐫𝐞𝐦𝐞𝐧𝐭𝐬? A data evolution problem. Addressed by UX features such as logical modelling interface, easily operable by both adept and non-technical data users. 𝐇𝐨𝐰 𝐭𝐨 𝐠𝐞𝐭 𝐚𝐧 𝐨𝐯𝐞𝐫𝐯𝐢𝐞𝐰 𝐨𝐟 𝐭𝐡𝐞 𝐠𝐨𝐚𝐥𝐬 𝐈’𝐦 𝐟𝐮𝐥𝐟𝐢𝐥𝐥𝐢𝐧𝐠 𝐰𝐢𝐭𝐡 𝐭𝐡𝐢𝐬 𝐩𝐫𝐨𝐝𝐮𝐜𝐭? A measurement/attribution problem. Addressed by UX features such as global and local metrics trees. ...and so on. You get the picture. Note that not only the active user suggestions but also the user’s usage patterns are recorded, acting as active feedback for data product dev and managers. This UI is like a product hub for users to actively discover, understand, and leverage data products while passively enabling product development at the same time through consistent 𝐟𝐞𝐞𝐝𝐛𝐚𝐜𝐤 𝐥𝐨𝐨𝐩𝐬 𝐦𝐚𝐧𝐚𝐠𝐞𝐝 𝐚𝐧𝐝 𝐟𝐞𝐝 𝐢𝐧𝐭𝐨 𝐭𝐡𝐞 𝐫𝐞𝐬𝐩𝐞𝐜𝐭𝐢𝐯𝐞 𝐝𝐚𝐭𝐚 𝐩𝐫𝐨𝐝𝐮𝐜𝐭𝐬 by the UI. How have you been solving the UX for your Data Products?