https://316c0611.personalpage-ahl.pages.dev/tags/e2e/ Hugo en https://316c0611.personalpage-ahl.pages.dev/posts/model-based-testing/ https://316c0611.personalpage-ahl.pages.dev/posts/model-based-testing/ Fri, 06 Jun 2025 14:05:26 +0200 <h2 id="intro"> Intro <a class="heading-link" href="#intro"> <i class="fa-solid fa-link" aria-hidden="true" title="Link to heading"></i> <span class="sr-only">Link to heading</span> </a> </h2> <p>What an exiting and strange time we live in! At the same time streets are poverty struck resembling void of beauty, and I can play with miniature special purpose machine brains sitting on gigantic legacy of modern computing. Words really is battlefront of light and dark. One of such brains can be constructed and used in app testing purposes.</p> <h1 id="every-user-flow-e2es---model-based-testing--state-machine-testing"> Every user flow e2es - model-based testing / state-machine testing <a class="heading-link" href="#every-user-flow-e2es---model-based-testing--state-machine-testing"> <i class="fa-solid fa-link" aria-hidden="true" title="Link to heading"></i> <span class="sr-only">Link to heading</span> </a> </h1> <p>is end boss dream of e2e testing. Covering every user flow is implementationally demanding. On top of it Teams claim e2es are demanding to maintain. So they opt out. Interestingly solution lies, as its often case in life, there where we don’t want to look at.</p> <p>With right code organization “every user flow” e2e testing is achievable and maintainable for almost the same effort as “some user flows” e2e testing.</p> <p>It’s very obvious why would you want it, so let’s dig straight into how.</p> <h2 id="setup"> Setup <a class="heading-link" href="#setup"> <i class="fa-solid fa-link" aria-hidden="true" title="Link to heading"></i> <span class="sr-only">Link to heading</span> </a> </h2> <p>The solution lies in graphs. Lack of general purpose graph libraries is the best tell how taboo graphs are. Yet anything done smart requires a graph. It is after all what powers AI as well. Enough introduction lets define a graph structure that will allow us to cover every user flow with effort comparable if not less than “some user flows”.</p> <blockquote> <p>UI state is node of the graph. (page, modal, panel - anything that requires user interaction)</p></blockquote> <blockquote> <p>User interaction needed to translate from one state to another is an edge of the graph.</p></blockquote> <p>Idea is very simple in fact. If you know about graphs. As soon as you have this graph you can traverse it in automated way! Thats the whole point. UI state/interaction graph is allowing automated traversal and each traversal is a user flow - aka e2e test.</p> <p>So test are generated from the graph with given parameters. No parameters = shortest path. If you want to go real fancy, real analytics data can be used as edge cost. This way tests can reflect real user behavior and be more realistic. But for now lets not go deep into traversal control.</p> <h2 id="user-interaction-library"> User interaction library <a class="heading-link" href="#user-interaction-library"> <i class="fa-solid fa-link" aria-hidden="true" title="Link to heading"></i> <span class="sr-only">Link to heading</span> </a> </h2> <p>No matter what you are testing if you are at home page and want to open a menu, same user interaction is used.</p> <blockquote> <p>User interaction library is defining set of edges</p></blockquote> <p>this promotes</p> <ul> <li>reusability</li> <li>maintainability</li> <li>automation</li> </ul> <p>Reasons why setting up “every user flow” doesn’t cost much more than “some user flows”. Or even less.</p> <h2 id="assertions"> Assertions <a class="heading-link" href="#assertions"> <i class="fa-solid fa-link" aria-hidden="true" title="Link to heading"></i> <span class="sr-only">Link to heading</span> </a> </h2> <p>Content assertions can be used to verify if components are rendered correctly. But this is sort of “glorified unit test” disguised as e2e test.</p> <ul> <li>e2es mimic user behavior</li> <li>user has no clue what should render</li> <li>he knows if data he needs is there</li> <li>if things glow green or red - do they work</li> </ul> <p>In e2e tests assertions can be split into intent libraries:</p> <p>One is to recognize the state of the UI. Another is to extract data from the UI.</p> <h3 id="recognizers"> Recognizers <a class="heading-link" href="#recognizers"> <i class="fa-solid fa-link" aria-hidden="true" title="Link to heading"></i> <span class="sr-only">Link to heading</span> </a> </h3> <p>Recognizers search for unique piece of UI to declare UI state.</p> <p>Now we see we in fact have two graphs! One is static - described so far. It is a prescription of how to use the app. But app has states so not all prescriptions are valid at all times. Meaning we must construct dynamic graph that represents actual state of the app. This graph can be populated via recognizers.</p> <p>Obviously these are also independent of the actual test goal. Fully reusable.</p> <h2 id="alternative-jargon--beginner-friendliness"> Alternative jargon / beginner friendliness <a class="heading-link" href="#alternative-jargon--beginner-friendliness"> <i class="fa-solid fa-link" aria-hidden="true" title="Link to heading"></i> <span class="sr-only">Link to heading</span> </a> </h2> <p>Graphs are, practice shows, a bit scary. So lets introduce less technical jargon</p> <p>static graph is experienced user. Experienced user knows how to do stuff in the app. Graph is a brain of a user. Dynamic graph is new user asking for advice how to do something in the app from experienced user.</p> <p>Once initial effort to setup graph traversals is done, devs write regular user interaction and assertions as they would otherwise laying them out in a set framework that defines the graph. Testing is then just declaring desired outcome (node/ui state ie something purchased) and giving required data. Test is generated</p> <h2 id="extractors"> Extractors <a class="heading-link" href="#extractors"> <i class="fa-solid fa-link" aria-hidden="true" title="Link to heading"></i> <span class="sr-only">Link to heading</span> </a> </h2> <p>Should be seen as “data extractors” - each component has an extractor that reads the data from the UI and then that data can be compared in automatic and dynamic manner leading into existing runtime validators of the app. Especially if <code>zod</code> is used. This once again allows to do more with less writing. Extractors are independent of the actual test goal. Reusable.</p> <h2 id="independence-test"> Independence test <a class="heading-link" href="#independence-test"> <i class="fa-solid fa-link" aria-hidden="true" title="Link to heading"></i> <span class="sr-only">Link to heading</span> </a> </h2> <p>If we look at the backend as a black box, we do not know if its stateless, stateful, or has some other state management. And we do not care. By assigning weights to edges, possibly based on analytics data, tests can have budgets and traverse in suboptimal ways. IE they can loop purchasing multiple times. In such scenario user flows should still work. Implicitly this is testing the backend for independence of implementation with respect to growing state.</p> <h2 id="intro"> Intro <a class="heading-link" href="#intro"> <i class="fa-solid fa-link" aria-hidden="true" title="Link to heading"></i> <span class="sr-only">Link to heading</span> </a> </h2> <p>What an exiting and strange time we live in! At the same time streets are poverty struck resembling void of beauty, and I can play with miniature special purpose machine brains sitting on gigantic legacy of modern computing. Words really is battlefront of light and dark. One of such brains can be constructed and used in app testing purposes.</p> <h1 id="every-user-flow-e2es---model-based-testing--state-machine-testing"> Every user flow e2es - model-based testing / state-machine testing <a class="heading-link" href="#every-user-flow-e2es---model-based-testing--state-machine-testing"> <i class="fa-solid fa-link" aria-hidden="true" title="Link to heading"></i> <span class="sr-only">Link to heading</span> </a> </h1> <p>is end boss dream of e2e testing. Covering every user flow is implementationally demanding. On top of it Teams claim e2es are demanding to maintain. So they opt out. Interestingly solution lies, as its often case in life, there where we don’t want to look at.</p> <p>With right code organization “every user flow” e2e testing is achievable and maintainable for almost the same effort as “some user flows” e2e testing.</p> <p>It’s very obvious why would you want it, so let’s dig straight into how.</p> <h2 id="setup"> Setup <a class="heading-link" href="#setup"> <i class="fa-solid fa-link" aria-hidden="true" title="Link to heading"></i> <span class="sr-only">Link to heading</span> </a> </h2> <p>The solution lies in graphs. Lack of general purpose graph libraries is the best tell how taboo graphs are. Yet anything done smart requires a graph. It is after all what powers AI as well. Enough introduction lets define a graph structure that will allow us to cover every user flow with effort comparable if not less than “some user flows”.</p> <blockquote> <p>UI state is node of the graph. (page, modal, panel - anything that requires user interaction)</p></blockquote> <blockquote> <p>User interaction needed to translate from one state to another is an edge of the graph.</p></blockquote> <p>Idea is very simple in fact. If you know about graphs. As soon as you have this graph you can traverse it in automated way! Thats the whole point. UI state/interaction graph is allowing automated traversal and each traversal is a user flow - aka e2e test.</p> <p>So test are generated from the graph with given parameters. No parameters = shortest path. If you want to go real fancy, real analytics data can be used as edge cost. This way tests can reflect real user behavior and be more realistic. But for now lets not go deep into traversal control.</p> <h2 id="user-interaction-library"> User interaction library <a class="heading-link" href="#user-interaction-library"> <i class="fa-solid fa-link" aria-hidden="true" title="Link to heading"></i> <span class="sr-only">Link to heading</span> </a> </h2> <p>No matter what you are testing if you are at home page and want to open a menu, same user interaction is used.</p> <blockquote> <p>User interaction library is defining set of edges</p></blockquote> <p>this promotes</p> <ul> <li>reusability</li> <li>maintainability</li> <li>automation</li> </ul> <p>Reasons why setting up “every user flow” doesn’t cost much more than “some user flows”. Or even less.</p> <h2 id="assertions"> Assertions <a class="heading-link" href="#assertions"> <i class="fa-solid fa-link" aria-hidden="true" title="Link to heading"></i> <span class="sr-only">Link to heading</span> </a> </h2> <p>Content assertions can be used to verify if components are rendered correctly. But this is sort of “glorified unit test” disguised as e2e test.</p> <ul> <li>e2es mimic user behavior</li> <li>user has no clue what should render</li> <li>he knows if data he needs is there</li> <li>if things glow green or red - do they work</li> </ul> <p>In e2e tests assertions can be split into intent libraries:</p> <p>One is to recognize the state of the UI. Another is to extract data from the UI.</p> <h3 id="recognizers"> Recognizers <a class="heading-link" href="#recognizers"> <i class="fa-solid fa-link" aria-hidden="true" title="Link to heading"></i> <span class="sr-only">Link to heading</span> </a> </h3> <p>Recognizers search for unique piece of UI to declare UI state.</p> <p>Now we see we in fact have two graphs! One is static - described so far. It is a prescription of how to use the app. But app has states so not all prescriptions are valid at all times. Meaning we must construct dynamic graph that represents actual state of the app. This graph can be populated via recognizers.</p> <p>Obviously these are also independent of the actual test goal. Fully reusable.</p> <h2 id="alternative-jargon--beginner-friendliness"> Alternative jargon / beginner friendliness <a class="heading-link" href="#alternative-jargon--beginner-friendliness"> <i class="fa-solid fa-link" aria-hidden="true" title="Link to heading"></i> <span class="sr-only">Link to heading</span> </a> </h2> <p>Graphs are, practice shows, a bit scary. So lets introduce less technical jargon</p> <p>static graph is experienced user. Experienced user knows how to do stuff in the app. Graph is a brain of a user. Dynamic graph is new user asking for advice how to do something in the app from experienced user.</p> <p>Once initial effort to setup graph traversals is done, devs write regular user interaction and assertions as they would otherwise laying them out in a set framework that defines the graph. Testing is then just declaring desired outcome (node/ui state ie something purchased) and giving required data. Test is generated</p> <h2 id="extractors"> Extractors <a class="heading-link" href="#extractors"> <i class="fa-solid fa-link" aria-hidden="true" title="Link to heading"></i> <span class="sr-only">Link to heading</span> </a> </h2> <p>Should be seen as “data extractors” - each component has an extractor that reads the data from the UI and then that data can be compared in automatic and dynamic manner leading into existing runtime validators of the app. Especially if <code>zod</code> is used. This once again allows to do more with less writing. Extractors are independent of the actual test goal. Reusable.</p> <h2 id="independence-test"> Independence test <a class="heading-link" href="#independence-test"> <i class="fa-solid fa-link" aria-hidden="true" title="Link to heading"></i> <span class="sr-only">Link to heading</span> </a> </h2> <p>If we look at the backend as a black box, we do not know if its stateless, stateful, or has some other state management. And we do not care. By assigning weights to edges, possibly based on analytics data, tests can have budgets and traverse in suboptimal ways. IE they can loop purchasing multiple times. In such scenario user flows should still work. Implicitly this is testing the backend for independence of implementation with respect to growing state.</p>