I am very curious whether chatGPT can help in software testing or even replace the role of a software tester. So today I checked to see if chatGPT can replace QA in software testing. The target under test is http://zero.webappsecurity.com/login.html. The figure below is the front page.
What is ChatGPT?
ChatGPT is a variation of the GPT (Generative Pre-training Transformer) language model developed by OpenAI. It is pre-trained on a large dataset of conversational text and can be fine-tuned for a variety of natural language understanding tasks such as dialogue systems and question answering.
How do I generate test scenarios and what is the result
In this fun experiment, I will make chatGPT to create a test scenario by writing a few sentences at the prompt.
Create a test scenario for the website’s front page
For the first experiment, I’ll have chatGPT create a test scenario for the target website. The prompt I’m using is “generate test case scenario for this site http://zero.webappsecurity.com/". And the result is shown in the attached image.
The result is quite good actually. It can write a test scenario systematically. The scenario is pretty basic. It only covers seen elements from a website. Even it creates a fictional flow. For example, clicking the “Online Banking” link will not redirect the user to the login page. But it can be a good start to understanding test scenarios and helping software testers write test scenarios.
Because the first experiment only results in a positive scenario, now for the second experiment, I’ll have chatGPT create a negative test scenario. The prompt I’m using is “generate negative case scenario for website http://zero.webappsecurity.com/". And the result is shown in the attached image.
The results are not quite different from the first result. But it was too focused on logging in.
Create a test scenario for the login page
For the third experiment, I will do something different. This time I’ll have chatGPT create a test scenario for the login page.
The prompt I’m using is “generate test case scenario for http://zero.webappsecurity.com/login.html".
ChatGPT is quite complete in making Test Scenarios. However, there are a few things to note:
- The scenario that is made is still limited to what appears on the screen. Not yet accommodate invisible flows, such as; failing to log in many times, logging in with saved cache, successfully login in then clicking the back button, etc.
- Create its fictitious pages. There is no “account summary page” page, let alone the contents of “account balance” and “transaction history”.
- The “Sign Up” button also doesn’t exist, let alone the “Sign Up page”.
Create a test scenario in cucumber
Then I have it to create a test scenario in cucumber.
Again, the result is very basic, even though it gives a disclaimer about it. But it can serve as a good template for testers to create test scenarios in cucumber.
Create a test script in Cypress
For the next experiment, I go further by having it create a cypress script.
I even try to produce the cypress script using a different prompt, and here is the result.
From the result, we’ll know that chatGPT only produce an example script and didn’t get the actual element. It needs a lot of tweaking for the script to work properly.
Then BOOM!!! It can’t handle it anymore.
So, I guess this is the end of my experiment. I already have a lot of fun doing it, and maybe will do it again in the future.
How to use AI to make an article
Another use of AI is it can help us to create an article in no time. One that I always use to help me make my medium blog is copy.ai.
CopyAI is a brand new tool that utilizes artificial intelligence to brainstorm and write the perfect copy for your ads, sales pages, social media updates, and pretty much anything else you need copy for. Forget having to brainstorm and write yourself... leave it to the bots.
If you are interested in using AI to make blog posts, rewrite sentences, or social media captions please take a look at the link below.
Conclusion
ChatGPT can create good, systematic test scenarios. Although the scenario is pretty basic and only covers seen elements, it can serve as a good start to creating the actual scenarios.
It needs a lot of adjustment to actually work properly with our software.
It can’t do the essential work of a software tester, which is to run the test.
0 comments:
Post a Comment