:::: MENU ::::

Software testing enthusiast.

 

ChatGPT

I am very curious whether chatGPT can help in software testing or even replace the role of a software tester. So today I checked to see if chatGPT can replace QA in software testing. The target under test is http://zero.webappsecurity.com/login.html. The figure below is the front page.

Front page of http://zero.webappsecurity.com

What is ChatGPT?

ChatGPT is a variation of the GPT (Generative Pre-training Transformer) language model developed by OpenAI. It is pre-trained on a large dataset of conversational text and can be fine-tuned for a variety of natural language understanding tasks such as dialogue systems and question answering.

How do I generate test scenarios and what is the result

In this fun experiment, I will make chatGPT to create a test scenario by writing a few sentences at the prompt.

Create a test scenario for the website’s front page

For the first experiment, I’ll have chatGPT create a test scenario for the target website. The prompt I’m using is “generate test case scenario for this site http://zero.webappsecurity.com/". And the result is shown in the attached image.

ChatGPT Result for a positive scenario

The result is quite good actually. It can write a test scenario systematically. The scenario is pretty basic. It only covers seen elements from a website. Even it creates a fictional flow. For example, clicking the “Online Banking” link will not redirect the user to the login page. But it can be a good start to understanding test scenarios and helping software testers write test scenarios.

Because the first experiment only results in a positive scenario, now for the second experiment, I’ll have chatGPT create a negative test scenario. The prompt I’m using is “generate negative case scenario for website http://zero.webappsecurity.com/". And the result is shown in the attached image.

ChatGPT Result for a negative scenario

The results are not quite different from the first result. But it was too focused on logging in.

 

Create a test scenario for the login page

For the third experiment, I will do something different. This time I’ll have chatGPT create a test scenario for the login page.

Zero bank login page

The prompt I’m using is “generate test case scenario for http://zero.webappsecurity.com/login.html".

Test scenario result for login page

ChatGPT is quite complete in making Test Scenarios. However, there are a few things to note:

  1. The scenario that is made is still limited to what appears on the screen. Not yet accommodate invisible flows, such as; failing to log in many times, logging in with saved cache, successfully login in then clicking the back button, etc.
  2. Create its fictitious pages. There is no “account summary page” page, let alone the contents of “account balance” and “transaction history”.
  3. The “Sign Up” button also doesn’t exist, let alone the “Sign Up page”.

Create a test scenario in cucumber

Then I have it to create a test scenario in cucumber.

Test scenario result for cucumber

Again, the result is very basic, even though it gives a disclaimer about it. But it can serve as a good template for testers to create test scenarios in cucumber.

Create a test script in Cypress

For the next experiment, I go further by having it create a cypress script.

Cypress script result

I even try to produce the cypress script using a different prompt, and here is the result.

Cypress script another result

From the result, we’ll know that chatGPT only produce an example script and didn’t get the actual element. It needs a lot of tweaking for the script to work properly.

Then BOOM!!! It can’t handle it anymore.

error

So, I guess this is the end of my experiment. I already have a lot of fun doing it, and maybe will do it again in the future.

How to use AI to make an article

Copy AI

 

 

Conclusion

ChatGPT can create good, systematic test scenarios. Although the scenario is pretty basic and only covers seen elements, it can serve as a good start to creating the actual scenarios.

It needs a lot of adjustment to actually work properly with our software.

It can’t do the essential work of a software tester, which is to run the test.

Connect with me

Krisnawan: Twitter | Linkedin | Medium

MydoQA: Twitter | Blog | Instagram | Facebook

0 comments:

Post a Comment

Find this blog interesting? Follow us