Jeroline
All right then uh hello everyone and welcome to yet another exciting webinar from the change leader series break free so today we'll be looking
Jeroline
At the topic mastering end-to-end test automation framework selection my name is Jeroline and I'll be your host for today so before we get started
Jeroline
I'd like to go over a few housekeeping rules your line is currently muted but don't worry you can submit your questions during the webinar
Jeroline
Through the chat option that you see at the bottom of your screen and we'll answer your questions as a part of the Q a
Jeroline
Section at the end of the webinar and please note that this webinar is currently being recorded and we'll send out the link of the
Jeroline
Recording through an email so that you can have a watch later or share it with your friends or colleagues so with that being said
Jeroline
Let me uh quickly share a bit about what Peak cloudy is and what we are trying to do to build aha moments for testers
Jeroline
And Developers next slide please uh Pcloudy is your Digital Playground that is equipped with all the tools required to test your apps on
Jeroline
The go with the most diverse set of real devices and browsers you can be sure of expanding your target audience across different geographies globally
Jeroline
Some of the core capabilities of a product include continuous testing devops Advanced automation AI based visual testing robotic process automation real-time test analytics and
Jeroline
Much more are cloud-based testing infrastructure hosts a rich diverse cloud with over 5000 plus real devices and browser combinations so if you want to
Jeroline
Know more about Peak cloudy uh do sign up on our platform and start exploring uh so now to share a bit about Break Free
Jeroline
Break Free was an idea that birthed out of the need for change in the testing domain we've seen the testing industry evolve and grow
Jeroline
Over the decades as a few leaders took the risk to break free from traditional approaches towards testing and come for the lot of new
Jeroline
Ideas and practices and they believe that this change can be accelerated at a much faster pace so we took the liberty of identifying and
Jeroline
Inviting a few change leaders for breaking the mold and embracing change to see the QA industry world and that's what breakthrie is all about
Jeroline
Our hope is that these sessions would spark ideas in each one of you to be the change in the QA industry for the industry
Jeroline
Itself to grow into greater Leaps and Bounds so with that uh let me welcome the change leader for today
Jeroline
Seasoned professional with over 13 years of experience in the QA industry and is currently with indeed Japan as senior QA automation engineer he has
Jeroline
A deep expertise in test Automation and integration with cicd platforms facilitating test processes and building the test strategy and life cycle of projects He
Jeroline
Is well versed in the agile scrub framework technical organization people management and all aspects of project and QA Management in addition to his technical
Jeroline
Skills misuth is also a frequent speaker in international conferences and has contributed to the testing Community by authoring books and insightful articles on testing
Jeroline
So without any further Ado let me hand it over to the speaker for today Misa
Mesut
Thanks thanks for the kind to introduction and welcome everybody yeah I hope this will be a productive session and if you have any questions
Mesut
Or comments or feedback just feel free to share so if it is interactive it will be more better and even more productive
Mesut
So as Jeroline told me a quality assurance engineer and I'm doing the quality assurance automation as well as most of my colleagues we are
Mesut
Doing the test automation as databases we know that quality assurance is not only testing there are lots of different Improvement rooms in the whole
Mesut
Software testing life cycle starting from the analysis of the requirements and then designing some test cases and executing them and finally doing the continuous
Mesut
Maintenance like analyzing the executions and collecting some evidences and trying to find some improvement rooms in the product as well so the aim for
Mesut
The quality assurances improving the team confidence to provide the best product for the end users or the clients so that being said quality insurance
Mesut
Is not only testing but testing and test automation is the we can say the heart of quality assurance because especially in terms of the
Mesut
Continuous testing if we don't do a test automation we cannot catch up with Comprehensive scope test automation is very important because in that regards
Mesut
We can do the iterative executions a lot of times a day because we are not having big releases or seasonal releases anymore but every
Mesut
Day we are having like small deployments or maybe we are raising a lot of merge requests so to ensure the quality or to ensure
Mesut
That those frequent chains do not break anything we should run the regression tests and of course doing that manually is not possible so in
Mesut
Short test automation is very important and today we will talk about test automation as well but what we will focus is selecting the best
Mesut
Options to build our testing framework test automation framework there are lots of different options so which one should I choose for example should I
Mesut
Basically choose Java language or maybe should I choose JavaScript and use some different Frameworks or maybe should I use any language and build my
Mesut
Framework from scratch without using any tools which is the best option so we will talk about this so I'll give you the answer and
Mesut
Let me tell you what I will tell at the end now the answer is there is no specific tool or library or a specific
Mesut
Option that I can recommend to you so the answer should depend on your requirements so what we should do is to analyze our requirements
Mesut
What kind of features we should Implement for example on what kind of browsers we should execute our test cases and lots of different decision
Mesut
Criteria we can think about and finally we can make our decision so what we will talk about is first I will make a quick
Mesut
Introduction uh I will list several options and then I'll share some rankings collected from some different sites and from different surveys and then I
Mesut
Would say kind of literature survey survey I will collect or I will share some collected information or comparisons from different studies and I will
Mesut
Share a little bit my personal experiences I implemented one test case on lots of different options and I will share my personal experience as
Mesut
A maybe we can think that as a developer experience and finally we will wrap up so let's get started
Mesut
First of all as I briefly explained in the beginning even starting from the programming language we can use or in what we can Implement
Mesut
Our test cases we can make our first decision so it can be either a Java JavaScript typescript python or anything else we can choose
Mesut
And that on top of that we should Implement our test cases and then we have the automated test cases we should execute them we
Mesut
Should run them so which means we need a test Runner and then after we are running the test cases of course we have to
Mesut
Assert that the X actual results are meeting the expected results so in the test scenarios you know we have some test steps and we
Mesut
Have some expected conditions this is the behavior or the specification of the product and to be able to assert that we need some assertion
Mesut
Libraries or maybe any other plugin that will help us to make those assertions and then maybe to do the API testing or the UI
Mesut
Testing we need some more libraries or plugins again like one of the very commonly used tools is selenium for UI testing and for API
Mesut
Testing on each program language we can use another option uh but instead of all these there are some kind of modern e2e test automation
Mesut
Frameworks those are already providing lots of different options built in with built-in libraries or the plugins for example let's say if you are using
Mesut
Uh Cyprus then I don't need to install any other external framework or library for the UI automation because I can already use the Cyprus
Mesut
Libraries or the metals for the functions already built in so we will talk about all these options and which one should I choose or
Mesut
While I'm choosing one of these what kind of decision criteria I can think about this will be the brief motivation of the talk
Mesut
So if some of the decision criteria we can think about is first of all the execution speed is very important because some of them
Mesut
Are running slower compared to the others some of them are running really fast and there are different types of executions for example we can
Mesut
Run in headless mode or which means we don't see any browser interaction at all it will be running in the background but we can
Mesut
Run in the headed mode as well which means we can see what is happening on the browser itself or a we can think about
Mesut
The ease of coding for example while using these different Frameworks which one is more user friendly like which one is more easy to use
Mesut
Or more easy to implement this is also very important in terms of the developer experience and flexibility for example while we are doing our
Mesut
Testing activities we are not running our test cases only on a specific platform for example not only desktop version but also the mobile versions
Mesut
Right because nowadays the products that we are developing are running in lots of different platforms the end users have mobile devices as well so
Mesut
It is not one of the PCS but sometimes the different mobile devices with the different screen sizes and different resolutions so how I can
Mesut
Easily scale to execute my test cases on different platforms or different versions how flexible those options are this also we can think about and
Mesut
Of course documentation and support is very important because whenever we have a problem or issue we should be quickly finding some solutions or Alternatives
Mesut
Provided by the community this is also very important and while using different features or the specifications of the Frameworks of course if they are
Mesut
Paid services or free services this will also decide or affect our decision and finally some kind of tricky features are not easy to implement
Mesut
On some of those options for example some of them are not supporting very well opening a new tab or a new page or whenever
Mesut
We are trying to handle the alerts or the pop-up Windows some of those options are not good enough to support the full functionality so
Mesut
This can be our decision criteria so as a warm up let me share some rankings collected from a site which is called the state
Mesut
Of JS so on this site there was a story being performed each year and in the last series it was done with the attendance
Mesut
Of I think almost 40K people and it was performed by key people driving this survey and they collect some information on different dimensions the
Mesut
First one is the retention retention means they ask people if they want to use this framers or the tools again and this ratio compared
Mesut
To the number of the all attendees attendees so we can see that for example some of them are the unit testing Frameworks and I
Mesut
Will mostly focus on the E3 testing Frameworks so for example we can see that playwright is doing really well it is 95 percent retention
Mesut
Rate it is very high and then we can see that Cyprus is 85 uh percent retention rate and Puppeteer and the other Frameworks are
Mesut
Going after those for example test Cafe and selenium does not have too much or too high retention rates as the others and then the
Mesut
Next Dimension they asked their next question or the category they asked for attendance opinions are the interest rates which means if they want to
Mesut
Learn these products how popular this product or the framers are in the community we can see that playwright is again very popular all most
Mesut
Of the people want to learn it and then again we can say a storybook and Cyprus and then coming the Puppeteer or a selenium
Mesut
Webdriver IO any other Frameworks including test coffee and then the next category is the usage which means if some people would use it again
Mesut
Or maybe they don't want to use it again but somehow they used this framework uh the the ratio of the number of these people
Mesut
In the total number of all data needs so we can see that again is high press and playwright are actually sorry this time pyrite
Mesut
Is not that much high but Cyprus is really high and commonly used a 42 percent of the whole people is somehow using or maybe
Mesut
Actively using or somehow once used this framework and then we can see the Puppeteer selenium and then playwright is coming because we can say
Mesut
That playwright is relatively new compared to the other I think Cyprus was released on maybe 2018-19 something like that if I'm not wrong but
Mesut
Playwright was released a few years later than that and the next category is the awareness which means how people know these Frameworks or the
Mesut
Tools we can see that Cyprus is very well known in the community and then storybook is really very known in the community and selenium
Mesut
Of course it is almost like 80 percent 77 I guess and the Puppeteer just mean a testing library and playwriters again a little bit
Mesut
Low rate because maybe due to uh still not major enough and relatively new uh but all of these framers we can say at least
Mesut
40 50 percent is known in the community and there's one more collected from Another Side which is the npm trends I put the last
Mesut
Six months Trends this is the download rates we can see that Cyprus is the mostly downloaded framework and then it is coming I guess
Mesut
Puppeteer and then uh it has selenium a web driver and a playwright core so there was a decent decrease uh by the end of
Mesut
The year at the end of December I guess maybe my guess it can be just a Christmas time but somehow it was decreased and
Mesut
Then the download rates would go to the normal levels Again by the start of the new year
Mesut
So after this some statistical information let's go over some comparisons like in this section I collected some evidences or some benchmarking studies from different
Mesut
People from different organization or studies so what they are doing is they're Head to Head making some comparisons and in most of the studies
Mesut
They are asking today again as a survey asking to the attendees as well and they are collecting the feedback from the audience as well
Mesut
So I will share some of them and then we will jump in to my personal experiences
Mesut
So the first study I want to share is uh that it was an event called the atleticoat speak it was performed by Philip rich
Mesut
And Andy Knight these guys are the ambassadors of different uh Frameworks the former one is for Cyprus and Andy for playwright and what these
Mesut
Guys are doing is they are having different categories for example for element interactions they are providing their Solutions and they are asking to the
Mesut
People by sharing Head to Head their Solutions which one is more maybe user friendly or maybe more feasible to the audience and then all
Mesut
The audience share their votings and then eventually they sum up they accumulate all the votes and we can understand which one was more acceptable
Mesut
By the users or the audience at least so what they asked is first of all they asked about the element interactions which was uh
Mesut
In that case in that regard which framework was more convenient to use and we can see that 62 percent of the audience said that
Mesut
Maybe Cyprus was doing better compared to the playwright and then going over the other questions for example dealing with iframes Cyprus was really weak
Mesut
In these terms and while waiting for elements to show up on the browsers it was almost same a 52 percent for Cyprus but for
Mesut
Alerts new pages and the languages they support and finally the browsers they support Cyprus could not do that much well compared to the playwright
Mesut
And for API testing corporations I'm not sure how we should consider this because most of the time we are separating the backend testing and
Mesut
The front-end testing right but if you somehow want to test both of them with the same framework we can consider this API testing capabilities
Mesut
As well so in short you can do API testing with both of them but according to all the audience to these events they said
Mesut
That maybe Cyprus was maybe more user friendly or more convenient to use in terms of the vacant testing API testing and for page object
Mesut
Model it was almost Same by the way Page object model is we want to encourage to use while using selenium and the other Frameworks
Mesut
Right but for the modern e2e test automation Frameworks a page object model is not recommended officially because they already designed their Frameworks in a
Mesut
Way that we don't need the page object models because we will a little bit talk about this but they are working asynchronously
Mesut
So sometimes a page of project models and waiting for the returns of the methods may cause some troubles so it is not recommended so
Mesut
In the terms the votes coming from the audience was exactly the same 50 percent
Mesut
And they made even the rematch of this event they made the second round and then there are some different questions as well for example
Mesut
How about developer experience how about finding selectors or test patterns and how about the login operations it can be in any other login approach
Mesut
Like providing the username or the password or any other credentials or dealing with the plugins or the execution East or execution speed and the
Mesut
Reporting I think I had one information regarding the execution speeds in the previous slide yeah I missed that so they said that they implemented
Mesut
The same test case under the same configuration and the result was playwright was running 4.6 times faster than Cyprus so this was the measurement
Mesut
By done over the same test scenario by these guys
Mesut
And another study this time this was shared by Michael Lynch on his post blog post and he tried to migrate the test cases as
Mesut
He implemented in Cyprus to declared so what he realized he made some comparisons first of all again which is very consistent with the previous
Mesut
Study he says that playwright is running five times faster than Cyprus he made some executions on both the local PC and the CI pipelines
Mesut
And the result was clear it was running five times faster and then he tried to implement some different features for example he tried to
Mesut
Assert some element attributes and then he tried to do some other assertions what he realized is in terms of the coding ears sometimes in
Mesut
Cyprus you have to wrap what you captured and then continue your next operations
Mesut
Because normally in playwright you can just add a late keyword to wait for the asynchronous operations but in Cyprus it is not possible to
Mesut
Do in that way so what you should do is first you should capture an element or any other operation and then you should wrap
Mesut
In the Cyprus format and then you should continue with your next text steps so there are a lot of sometimes there are lots of
Mesut
Different inner Loops in Cyprus because in play that you can just
Mesut
Implement your lines one by one by adding just a weight keyword in the beginning of the line but in Cyprus it is a little
Mesut
Bit more complicated to do that so in terms of coding is here says playwright was doing a little bit better than Cypress and he
Mesut
Investigated the support from the developers or the maintainer people of these tools and there were more open issues in Cyprus repository he realized
Mesut
And finally in terms of the resource allocation in CI because you know when we push our code normally we execute in the virtual machines
Mesut
In the CI pipelines and sometimes we are running in Docker images so first of all the docker image should be downloaded with the playwright
Mesut
Or the Cyprus installed inside and then we can execute our test cases in that virtual machine so downloading this machine as well the download
Mesut
Process itself regardless of the execution of the test cases will be slower in Cyprus because he said that the Cyprus images 940 megabytes where
Mesut
The playwright images only 650 megabytes so I'm trying to avoid saying anything subjective but of course all the framers have different strengths and different
Mesut
Weaknesses and as I told in the beginning there is no clear answer so there are lots of different dimensions we can think about but
Mesut
There are some obvious things as well for example if the pit is important for me obviously it seems that playwright is running faster than
Mesut
Cyprus but for example in terms of ease of coding as well play right they might be a little bit better but of course Cyprus
Mesut
Has some other strengths as well so we can we will make a final summary at the end don't worry I will remind all the
Mesut
Key points of all the Frameworks again this is a another study just done by someone else namely Quarry house and uh the same personality
Mesut
Was implemented in two different options and we can see that again the playwright is a little bit more easy to understand because he just
Mesut
As we just discussed he just added keyword in the beginning of each line and he doesn't have to add some more rep code
Mesut
And in the scenario what we realize is there's a cross-domain issue as well what means it starts with a URL and in the middle
Mesut
Of the test case the test jumps into another URL so it is going to navigate to another domain which we call as close domain
Mesut
So in this terms Cyprus did not support this feature at all but recently they added this feature but still it is a little bit
Mesut
Complex and complicated so in playwright you can just continue your assertions when you click the website is navigating to another URL but you can
Mesut
Just do your assertions to assert that the link is properly set but in Cyprus it is not that much sumut so you have to
Mesut
Run the origin code to do the cross domain operations
Mesut
So I will a little bit speed up to quickly go because there are some duplications in these studies which means these studies are consistent
Mesut
And of course they might Implement different scenarios and they make their they can make their executions in different environments but most of the information
Mesut
Is consistent so this another study was done in my organization in India with by some teams and they share some similar information and in
Mesut
Terms of the sorry execution speeds they relate this to the actuality working principles they say that the selenium for example is working with the
Mesut
Web driver protocol this is why it is it tends to execute fast but the playwright is running with the Chrome debugger protocol but these
Mesut
Two is running a purpose apart from the browser as well but Cyprus is running on the same browser so it is just running some
Mesut
Scripts so this it tends to run a little bit slower compared to the other other options
Mesut
So summing up all the information compared uh playwrights with Cyprus we can say that especially based on some features play uh play artist blink
Mesut
Really well but Cypress is having some difficulties especially supporting the iframes or the file operations for example if you want to download file or
Mesut
Upload file or managing the alerts or opening new tabs in new page Cyprus is having some difficulties and especially in terms of the speed
Mesut
We can see that playwright is running five times faster but for the other things there are lots of different similarities for example on both
Mesut
Of them we can generate some HTML reports showing the execution details like execution duration on each individual test case or for example whenever a
Mesut
Test case fails we have a chance to retry in both options so and we can even capture the screenshots or even the video in
Mesut
Both of them and the configuration management is very well in both options so we can just introduce lots of different environmental variables and different
Mesut
Configurations in the config files hmm but again just for ease of coding I wanted to open a another parenthesis this is one important aspect
Mesut
And the other is the important aspect on the other hand is this time not use of coding but ease of understanding the code so
Mesut
The readability of the code so this time I can be a little bit subjective my personal opinion is Cypress is a little bit more
Mesut
Easy to understand for example again this is one quote I implemented in both options so if I just read the line for example on
Mesut
The right hand side there's a Cyprus implementation uh for example I will read one of the lines which says side that get one of
Mesut
The elements and then it should be disabled it is just like the natural language that we are using it is just the English sentence
Mesut
So I can straight away they understand what it is doing it is just getting some elements and it should it tries to assert it
Mesut
The Element should be disabled that's all I don't have to worry about understanding the code or what is being done by this code
Mesut
So after this playwrights and Cyprus comparison this was the longest one because mostly I can say that these two are the main locomotives of
Mesut
The Frameworks uh used mostly used in the community but we can quickly go over some different comparisons as well for example test Cafe versus
Mesut
Cyprus we can see that lots of different execution details here it was again done in one team in my organization and what we can
Mesut
See is the execution speed varies when we run in our local environment versus when we're running our ACI pipelines so this may depend on
Mesut
Lots of different things for example even the initialization of the test Runner can be different in these different options this is why maybe running
Mesut
In local environment Cyprus is taking less time compared to test Cafe while running from gitlab Cyprus is taking really long time compared to of
Mesut
It and again for some feature supports especially iframes or handling multiple windows or tabs we can still see that the Cyprus is having a
Mesut
Limited support but the test Cafe is having a well support on these features by the way uh of course every day they are trying
Mesut
To improve these products and the Frameworks for example Cyprus didn't support executing the test cases on Safari at all until I guess the end
Mesut
Of last year but they now are supporting but it is still kind of pilot usage so even if they you want to use
Mesut
Safari executions the in Cyprus they are still using play right on the background so you can somehow execute your test cases on Safari or
Mesut
On webkit but they just recently added this support So previously it was not supported at all but what I'm trying to say is even
Mesut
If some of the features are not supported yet they might be improved in the future and they can be added in the feature in
Mesut
All these Frameworks and when we check the selenium versus a Cyprus comparison we can see another event very similar event the as I showed
Mesut
In terms of the Cyprus versus playwright this time again the the same event the series which was let the code speak this time performed
Mesut
By Angie Jones and the call before these guys again are the ambassadors of these Frameworks and we can see in some aspects Cyprus was
Mesut
Doing really well compared to selenium but especially again related or a uh directly attached to some feature supports selenium was providing some more support
Mesut
Compared to the Cyprus and again the same event series performed another episode by different people this time again Angie for selenium and again Andy
Mesut
For playwright and the most of the majority of the votes coming from the audience was endorsing playwright so the result of this match was
Mesut
The win for players and again we can see some information regarding this comparison and they share that Which languages they are supported by these
Mesut
Framework and which test Runners we can integrate or we can use as a plugin while we are using these Frameworks but in terms of
Mesut
These aspects they are doing really very similar to each other so only on specific aspects they have some differences but of course the main
Mesut
Functionality and the main features are already provided by both options
Mesut
So without any further Ado let me share my personal experiences so so what I have done after the this literature survey I wanted to
Mesut
Use those uh Frameworks by myself I wanted to implement and I wanted to feel how the user experience or how the usability or the
Mesut
Understandable or the ease of coding is uh they are doing on each of these options so what I have done is I selected in
Mesut
One I selected one test scenario which was already implemented in my automation set so what I did is I implemented this scenario in with
Mesut
All the other options as well in addition to the solution that I was already using so this scenario is very simple one which is
Mesut
Basically going to the one URL base URL and then normally when I first go to the web page then I have a button but
Mesut
At first it says disabled so right after going to the URL I'm doing an assertion to check that the button is disabled and then
Mesut
After I type some queries then I should see that the button is becoming enabled after providing my query so this was just a job
Mesut
Seeking web page so users can navigate to this webpage and do some search for some jobs in different areas for example in Japan let
Mesut
Me find the engineer related jobs right so I can provide a keyword which is the engineer and I can provide a location like find
Mesut
The jobs in popular and I can click the button to perform the search this is the scenario and then after I clicking the button
Mesut
Of course I'm supposed to go to another page and I should see the search results but of course I have some other assertions after
Mesut
Clicking the button which is checking the URL if it has a proper set with the keyword that I provided and in the new page
Mesut
The query field is still there so I will make an assertion if it is including the keywords that I already provided in my previous
Mesut
Test step so in this scenario there's no complicated feature like there is no alerts there's no iframe there's no drop down menu and there's
Mesut
No cross domain everything is working on the same website so it is a pretty straightforward but I have some key aspects for example first
Mesut
Of all it is very important to run this on desktop and the mobile version as well because in Japan most of the users love
Mesut
Using mobile devices and they allow especially using IOS operating system and which means the Safari browser so I especially Safari execution was very important
Mesut
For me and for testing the mobile version the application that I was used using was not responsive which means if I change the window
Mesut
Size I was still not able to test the mobile version so what I had to do is I had to add a custom header
Mesut
To my request which is adding a user agent information to the header in that regard by this way I can mock the browser as
Mesut
I'm using from a mobile device so customizing the requests in my execution was very important for me so if those Frameworks are flexible enough
Mesut
Enabling me to perform this step was really a crucial decision criteria for me and of course all the other aspects like ease of coding
Mesut
The code understandability and the executions a bit and all the other aspects so after all I Implement all this I realized that I can
Mesut
Pass lots of different execution arguments so my execution command as well and in the meantime of course I can utilize the config file to
Mesut
Pass some different configurational variables as well and I made a feature metrics like which one is supporting which feature or which is not supporting
Mesut
So as a summary in short uh by introducing to introduce some different custom Fields into my requests to customize the header I realized that
Mesut
By selling moment Network it is possible I'm not saying it is not possible but there was not a very straightforward way to do that
Mesut
So you have to deal with the browser capabilities or the execution options and it is not that much straightforward but in the other Frameworks
Mesut
Like Cyprus or test of it there are already some built-in methods so you can utilize so it was easier to use in that trigger
Mesut
This was the first criteria and then running the Headless again in selenium you have to deal with the browser options and the browser capabilities
Mesut
But for the other one it is just an Aflac that's all you don't have to do anything else and Safari at that time when
Mesut
I did this study again the Cyprus did not support Safari at all but now that's kind of supporting it is a kind of a
Mesut
Possible to execute on the suffer and for reporting the results everything all these options are given the chance to generate some HTML reports but
Mesut
Especially Cyprus this is one thing I love about Cyprus it has a dashboard and it is super easy to use it is even giving
Mesut
A chance to track the flatness you can understand which test days are lucky on the dashboard and you can track lots of different execution
Mesut
Details and taking the test cases like in other Frameworks it is called as a annotation or marker we can say in Cyprus you can
Mesut
Add text to all the individual test phases and even before starting the execution for example in the before who you can already read the
Mesut
Text of the tests so for example by running this command this that any specific attack is set to something you can basically skip even
Mesut
Before starting the test sets so it is giving a lot of chance in terms of the flexibility and for other debugging purposes or the
Mesut
Retry mechanisms most of the Frameworks are having chance to modify this execution options
Mesut
And of course I did some measurements as well but it does not have a scientific just a couple of execution results in different environments
Mesut
So again it was very similar to the observations made by the other studies
Mesut
So I will try to wrap up and then we can discuss if there is any question coming from the audience or if there is
Mesut
Any feedback or any other opinion I would be really very glad to hear your ideas or maybe we can even ask if some people
Mesut
Are using any specific options or they are using some other things and maybe they have some recommendations so but first of all the summary
Mesut
So what we have talked about we have gone over lots of different options but I'm not saying this one is the best option because
Mesut
The key idea of us first of all clarifying the requirements which features we should need or in what kind of execution environments we should
Mesut
Run our test cases and based on these requirements we can do our decisions but every few key aspects of all these options are for
Mesut
Example Cyprus it lacked support for some specific features this is one weakness of Cyprus as I realized but one real strength which is in
Mesut
Place in Cyprus is its dashboard and smart parallelization is SEI you can run all the tests in parallel but you can divide the switch
Mesut
In the spec file level for example there are lots of different spec files some of them you can assign to a machine and for
Mesut
Some others you can assign to another machine but in Playland for example you can still do the parallelization and even within a spec file
Mesut
If you have more than one test case you can split the tests within the spec file as well in the individual test case level
Mesut
So player implanticalization is even more major compared to the separates
Mesut
And one more very important aspect or the strength of player advice the device emulators so if you use any of the device emulator you
Mesut
Don't have to do the custom headers uh Edition as well you can just use the mobile version and for selenium and night watch some
Mesut
Features were not straightforward to use but they are really lightweight and fast and understandable especially night watch but of course there are some different
Mesut
Aspects as the documentation and the community support as well so please make your clarifications first of all know what you need what kind of
Mesut
Test you will Implement what kind of execution envirus you will have and then after a comprehensive study comprehensive Benchmark do your decision this is
Mesut
A call for action and I added all the references that I shared because I shared lots of different literature survey you can find all
Mesut
Of them in the slides and thanks for listening again I will be more than glad to answer the questions if there are any
Mesut
Jeroline are you there you can you help me to go over the questions or I think there are already some in the QA q
Mesut
A session so yeah okay so I can start going over them one by one the first question is Cyprus used as API automation yes
Mesut
You can use you can basically send your request and collect the responses and then a parse the response code and the response body and
Mesut
To your assertions you can use the second question is the survey is related to JS ecosystem yes JavaScript and typescript these two uh but
Mesut
Not the others I I think this is a relate this question relates to the first survey that I shared yes it is Javascript Plus
Mesut
Typescript next question does Planet use for API testing yes again yes you can do these Frameworks are focused Muslim UI testing okay mostly on
Mesut
Front-end testing but still you can do the backend testing as well
Mesut
Next question which one is the best for react web app playwright Cypress selenium again same message it depends I mean the system that you
Mesut
Are testing the system under tests what kind of features it has or for example you have cross domain or not so what I can
Mesut
Say is of course you can test any react web app or any other application so it is a little bit kind of both related
Mesut
Or and not in the same time so whatever you are testing the test framework itself can be decided regardless of the product that we
Mesut
Are testing because as a tester most of the time we are doing our testes or testing activities in the Black Box manner I don't
Mesut
Care about the implementation details I don't care which language they used or in what way they implemented paper scenarios but I care about the
Mesut
Behaviors what kind of specifications they have what kind of behaviors or the expected results I have so it is I can make this decision
Mesut
Regardless of the product code and the technology they are using but in the meantime I can relate because sometimes I can make a decision
Mesut
In which a I'm deciding the technology I will use to implement my test port because sometimes I want to share the implementation effort with
Mesut
The development team as well so if I select a total different language or the technology from the development teams sometimes it will be a
Mesut
Little bit difficult to share for example they already are familiar to use typescript language and then I didn't care about the development team and
Mesut
I just select python right so after some time if I ask them to implement some test cases they will say that we are using
Mesut
Typescript but you didn't ask us and you select python so now we can take them out so let us both related and not to
Mesut
The production code I could say next question can we migrate an existing python framework to a player oh yeah definitely definitely and I can
Mesut
Even say that nowadays AI is a very hot topic right so aim machine learning techniques there are even some initiatives which are doing this
Mesut
Migration automatically so they are just reading the existing repository and trying to migrate to another tool or framework by just running AI algorithms of
Mesut
Course it might fail in some scenarios I'm not saying it will do 100 but either in automated or manual manners it is definitely possible
Mesut
To migrate and actually I might get my framework from Cypress to player
Mesut
I think I answered this so the next question is does Cyprus or playwright support mobile app automation yes I mean if the answer is
Mesut
A native application the answer is no but what we can do is it is the mobile version of the browser so I guess it
Mesut
Is uh I would say it is enough in terms of automated testing efforts because trying to do the native app testing itself is really
Mesut
Very cumbersome and you you need to implement lots of different things but if you just test the mobile version of the browser and then
Mesut
Maybe a little bit to manual testing on the native app itself it might be enough in terms of the testing activities because eventually we
Mesut
Will already need some manual testing right whatever we are automating or even if we automate all the testes hundred percent automation we will still
Mesut
Do the manual testing so that automation part I guess we can do the a mobile browser version I would say
Mesut
Next question how TD and BTD Frameworks work in Cyprus for this I don't have any personal experience but I know that there are some
Mesut
Plugins to utilize tdd and bdd Frameworks but again I won't say any specific statement on this because I don't have any personal experience but
Mesut
I know that it is somehow possible to integrate these approaches
Mesut
Next question for a complete beginner is test automation which one would the best to learn Cypress or Salem
Mesut
Maybe it will be again a personal statement but as a complete beginner maybe I will recommend sentence because there are some already built-in functions
Mesut
So you don't have to build everything or Implement everything from scratch so Cyprus could be easier to start from scratch for a complete
Mesut
Next question well there are lots of questions thanks a lot uh whenever time is available we can go over all of this so next
Mesut
Is which is the best testing tool for websites and apps according to you
Mesut
I think we already answered this it depends okay but I I will say my uh personal uh opinion on this lately I'm a more
Mesut
Attending to use players because they are trying to cover most of the features
Mesut
But at first initially I was implementing Cyprus in Cyprus but then I started using clearight as well especially giving up from Cyprus dashboard was
Mesut
Not easy for me I really like that feature in terms of Cypress functionality
Mesut
Collaborative cucumber framework this is related to the other bdd question I'm not very sure I know there are some bdd plugins especially cucumber framework
Mesut
I'm not sure about this let's investigate this later do you have any suggestion for Tableau reports testing W reports are available in web version
Mesut
Also again I didn't have any personal experience on this but it is just the web version uh it should be possible to do by
Mesut
These means but again thinking about the drop down menus and iframes maybe it would be better to use playwright again instead of Cyprus
Mesut
Which one is the best for UI automation we are working on the e-commerce domain and we have 30 plus websites currently we are using
Mesut
Selenium for UI function automation we need to do UI automation visual testing please suggest me one is better all right so
Mesut
I think in terms of visual testing I could recommend Cyprus because I know that they have some packages and libraries especially in terms of
Mesut
Visual testing so you could search for how to use Cyprus to do visual testing activities
Mesut
Which framework is best suitable to automate ml scenarios wow msnr I mean I don't know because
Mesut
I mean if we are talking about ml testing it is a completely different area right because it is not even we are not talking
Mesut
About even a web app or even a backend so testing ml is completely different and I'm not sure we should make a decision among
Mesut
These options in terms of ml testing any idea whether players support performance testing I don't think so again performance testing is something totally different
Mesut
So there are a lot of different options to do that just especially for this purpose they are implemented like there are J meters or
Mesut
Others tools and libraries implemented just for performance testing purposes but the main motivation of these modern E3 test automation Frameworks is not a performance
Mesut
Testing but insert functional testing so maybe you should you could somehow do that maybe but I would say it would not be the best
Mesut
Solution to do the performance and stuff next question your thought on python Robot Framework I like robots framework it's really cool
Mesut
And uh especially if you are working in robotics domain I'm living in Japan and before indeed I was working for a robotics company and
Mesut
There especially we are using a Ros which is robotics operating system especially if you are using that kind of specific product specific system under
Mesut
Test uh Robot Framework could be really a good decision
Mesut
But not necessarily you can use with any application under test but especially if you are using that kind of specific product it might be
Jeroline
A good decision I think I'm at the end of the question list all right uh thank you so much Mrs for answering those questions
Jeroline
And for the great presentation of course it was a pleasure to have you with us today and I'm sure our audience gained a deep
Jeroline
Understanding of end-to-end test automation Frameworks and insights you uh and the insights that you shared today will help them make an informed decision in
Jeroline
Their upcoming projects and if we ran it if you don't have any more questions uh then that's all for today's webinar thank you all
Jeroline
For attending and being a being an interactive Bunch today and we hope you found the webinar to be valuable and uh informative remember keep
Jeroline
Breaking free and pushing the boundaries of what's possible in the QA industry see you see you at the next break free webinar have a
Mesut
Great day thank you for organizing this it was a wonderful event thank you okay
