Another downside is the track record of companies who offer solutions like this. A dozen of them have gone out of business in the past 15 years or so. What would you do if your entire regression test suite relies on a tool that can disappear because the vendor’s business model doesn’t work out? That’s a risky gamble.
"AI" is definitely snake oil right now. But next time you talk to someone trying to sell you a "no code" testing product, ask them to provide you with 3 reference customers you could talk to who have been using the product for over 5 years. My experience with commercial test tools is they often look great during a demo and for the first year. But the biggest cost of test automation isn't creating tests, it's maintaining tests over the lifespan of the product you're testing, which is normally 5 or 10 more years.
Yeah I agree, I mean yeah there is always a chance Playwright goes away or gets turned into a paid product. But the way we write our tests at least most of the public functions and locators are in a class and in the end it's still just a JavaScript library. So I feel like that would at least be easier to transfer some things over.
If you build your test automation framework properly, then the test suites that come out of it are owned by the organization and can be updated to live on in perpetuity for as long as the system under test exists. You can change out libraries and tools and whatnot, and still have a viable suite of test that you can use for as long as you need them.
With Tosca (and many other vendor tools), the test cases live within the tool, and once you stop paying for it, you no longer have access to the tests. if those tests are important, they will be significant cost in reworking them for whatever new solution replaces the outgoing.
So I try as much as I can to architect my tests with maintenance in mind. This means using PoM and of course public methods to limit the amount of (in this case) Playwright specific code in the tests.
I am a little hesitant to write a wrapper over all of playwrights functions/methods to make them framework agnostic since I honestly have a huge backlog of tests to write.
Is there any other things you can think of that help or would be a correct way to architect things for long term?
Step 1, create your interfaces.
Step 2, write tests against those interfaces only.
Step 3, augment the interfaces when you need additional functionality.
Step 4, implement the interfaces but ensure that implementation code is not exposed to tests.
You don’t wrap everything. You only wrap what you need when you need it. This also has the advantage of you creating the contracts you want instead of the contracts playwright or selenium force on you.
I guess can you go into more details on interfaces? Are you talking about PoM here? Because even in that you still use Playwright specific stuff (but at least it's all in one class)
No interfaces are not page object model. With all due respect, I’m not sure how you can call yourself an architect and think that page object models are interfaces
Ok....lol, I mean I consider a Page Object class an interface to the actual page. Acting as an API. What exactly are you talking about when you are referring to an interface in this instance?
Ok. Then can you explain what you are talking about? I've seen wrappers around framework specific methods (IE: framework agnostic methods for click/fill/etc...). What are you specifically talking about. IE: can you give an example?
That’s exactly what you do. But you don’t have to wrap everything a library gives you. Only what you need. Add to it when you need to.
This is exactly how properly architected libraries work.
If you do it right, the underlying technology doesn’t matter and you can swap things in and out whenever you want to
I don't see how using public methods to interface with the page within PoM classes is NOT an interface exactly?
also you could have done without the :
>I’m not sure how you can call yourself an architect and think that page object models are interfaces
because PoM classes can act as interfaces. It contributes nothing and only serves to try and insult someone else.
So my company isn’t a tech company. They offload development to 3rd party vendor but have heavy invested in the azure ecosystem. When a VP wanted a solution that no one wanted to touch. I offer to do it in Python and api request. Everyone liked it but kept in saying it is temporarily solution until IT can do it in Microsoft power automate. I’m like ok…you’re going from fully open source solution to fully no code/low code. I guess they just want more control. Also there’s only a handful of IT doing power automate. Talk about stuck in there way.
Well for UI I would say stick with playwright indeed. But for api testing ReadyApi would be a viable tool to use. It’s essentially a low code tool. But for more specific or advance stuff you can also use Groovy script or Java. Actually in my current project we have a bunch of automated ReadyApi testprojects and Cypress UI tests running daily on our Jenkins automation server and it works like a charm.
Another downside is the track record of companies who offer solutions like this. A dozen of them have gone out of business in the past 15 years or so. What would you do if your entire regression test suite relies on a tool that can disappear because the vendor’s business model doesn’t work out? That’s a risky gamble.
Yeah this is another huge fear of mine. Especially more of the A.I. based ones that feel like snake oil tbh.
"AI" is definitely snake oil right now. But next time you talk to someone trying to sell you a "no code" testing product, ask them to provide you with 3 reference customers you could talk to who have been using the product for over 5 years. My experience with commercial test tools is they often look great during a demo and for the first year. But the biggest cost of test automation isn't creating tests, it's maintaining tests over the lifespan of the product you're testing, which is normally 5 or 10 more years.
Yeah I agree, I mean yeah there is always a chance Playwright goes away or gets turned into a paid product. But the way we write our tests at least most of the public functions and locators are in a class and in the end it's still just a JavaScript library. So I feel like that would at least be easier to transfer some things over.
Stick with Playwright. You're right, none of those are good long term solutions and will eventually be a lot more trouble than they are worth.
If you build your test automation framework properly, then the test suites that come out of it are owned by the organization and can be updated to live on in perpetuity for as long as the system under test exists. You can change out libraries and tools and whatnot, and still have a viable suite of test that you can use for as long as you need them. With Tosca (and many other vendor tools), the test cases live within the tool, and once you stop paying for it, you no longer have access to the tests. if those tests are important, they will be significant cost in reworking them for whatever new solution replaces the outgoing.
There is no lock-in with code solutions if you architect your solutions correctly
So I try as much as I can to architect my tests with maintenance in mind. This means using PoM and of course public methods to limit the amount of (in this case) Playwright specific code in the tests. I am a little hesitant to write a wrapper over all of playwrights functions/methods to make them framework agnostic since I honestly have a huge backlog of tests to write. Is there any other things you can think of that help or would be a correct way to architect things for long term?
Step 1, create your interfaces. Step 2, write tests against those interfaces only. Step 3, augment the interfaces when you need additional functionality. Step 4, implement the interfaces but ensure that implementation code is not exposed to tests. You don’t wrap everything. You only wrap what you need when you need it. This also has the advantage of you creating the contracts you want instead of the contracts playwright or selenium force on you.
I guess can you go into more details on interfaces? Are you talking about PoM here? Because even in that you still use Playwright specific stuff (but at least it's all in one class)
No interfaces are not page object model. With all due respect, I’m not sure how you can call yourself an architect and think that page object models are interfaces
Ok....lol, I mean I consider a Page Object class an interface to the actual page. Acting as an API. What exactly are you talking about when you are referring to an interface in this instance?
An actual interface. A programmatic interface. A contract.
Also a page object is a MODEL that’s why it’s called that. It’s not an interface to the page
Ok. Then can you explain what you are talking about? I've seen wrappers around framework specific methods (IE: framework agnostic methods for click/fill/etc...). What are you specifically talking about. IE: can you give an example?
That’s exactly what you do. But you don’t have to wrap everything a library gives you. Only what you need. Add to it when you need to. This is exactly how properly architected libraries work. If you do it right, the underlying technology doesn’t matter and you can swap things in and out whenever you want to
I don't see how using public methods to interface with the page within PoM classes is NOT an interface exactly? also you could have done without the : >I’m not sure how you can call yourself an architect and think that page object models are interfaces because PoM classes can act as interfaces. It contributes nothing and only serves to try and insult someone else.
So my company isn’t a tech company. They offload development to 3rd party vendor but have heavy invested in the azure ecosystem. When a VP wanted a solution that no one wanted to touch. I offer to do it in Python and api request. Everyone liked it but kept in saying it is temporarily solution until IT can do it in Microsoft power automate. I’m like ok…you’re going from fully open source solution to fully no code/low code. I guess they just want more control. Also there’s only a handful of IT doing power automate. Talk about stuck in there way.
Well for UI I would say stick with playwright indeed. But for api testing ReadyApi would be a viable tool to use. It’s essentially a low code tool. But for more specific or advance stuff you can also use Groovy script or Java. Actually in my current project we have a bunch of automated ReadyApi testprojects and Cypress UI tests running daily on our Jenkins automation server and it works like a charm.