辅导 program编程、讲解 python程序设计

Part 1: Coverage
Introduction
This component requires the development of a comprehensive tool to analyse a provided test suite for
a piece of software. The analysis must cover two key white-box testing metrics: statement coverage
and branch coverage. The aim is to assess the efficacy and thoroughness of the test suite in detecting
faults and ensuring robustness in the software.
Objectives
• Implement a tool that takes a series of given test inputs and runs them on a program.
• Report the statement coverage and branch coverage for the program when run using the series
of test inputs.
Requirements
1. Statement Coverage
Objective: Determine the percentage of executable statements in the software that are executed by
the test cases in the test suite.
2. Branch Coverage
Objective: Identify and report the number of branches through the program’s control flow graph that
are covered by the test suite.
Input Specifications
Your program should take 2 command-line arguments:
1. The path to a Python script
2. The path to a directory containing a set of input (.in) files
It should be called using the following command:
python coverage.py
Output Specifications
Your program should produce output indicating:
1 Statement Coverage: The count of statements executed during testing.
2 Branch Coverage: The count of intra-procedural paths executed during
testing.
For example:
1 Statement Coverage: 150
2 Branch Coverage: 19
Page 3
Part 2: Fuzzing with Mutated Inputs
Introduction
In this part of the assignment, you will develop a fuzzer designed to automate the generation and
mutation of test inputs to maximise the branch coverage of a test suite. The primary goal is to expand
the test coverage by identifying and adding inputs that expose new branches in the software under
test.
Objectives
• Develop a fuzzer capable of generating and mutating test inputs.
• Implement a method to measure the increase in branch coverage.
• Automate the process of enhancing the test suite with inputs that increase branch coverage.
Requirements
This task requires you to take a program along with a series of inputs and mutate the inputs to achieve
a minimum branch coverage (note that in Part I we ask for statement coverage and branch coverage).
You must automatically improve the test suite by adding mutated inputs that increase the branch
coverage.
Implementation Specifications
• Use the fuzzer to apply mutations to the initial set of inputs.
• For each mutated input, execute the test suite to determine if the mutation results in increased
branch coverage.
• If an input increases branch coverage (by reaching new conditions not previously tested), add
it to a ’population’ of effective test inputs.
• Continue this process until no further increase in branch coverage is observed, aiming to achieve
the largest possible branch coverage.
• Write the final set of test inputs that collectively provide the highest branch coverage observed
to a file.
Input Specifications
Your program should take 2 command-line arguments:
1. The path to a Python script
2. The path to a single text (.in) file
It should be called using the following command:
python mutation_fuzzer.py
The text file will contain a set of inputs, each on a new line.
For example:
1 Never
2 Gonna
3 Give
4 You
5 Up
Page 4
Output Specifications
Your program should write back to the provided input (.in) file with exactly the same number of
input strings as was provided initially.
For example:
1 Never
2 Gonna
3 Let
4 You
5 Down
Part 3: Grammar-Based Fuzzing
Introduction
Grammar-based fuzzing is a commonly used method to test programs that consume structured inputs,
particularly input parsers.
Objectives
• Implement a grammar-based fuzzer to generate structured inputs for testing.
• Explore various grammar structures to hit or exceed a branch coverage threshold specified.
Requirements
This task requires implementing a grammar-based fuzzer capable of generating structured inputs
based on a specified grammar. The goal is to hit or exceed a branch coverage threshold by generating a test suite that effectively tests the target program.
Implementation Specifications
• Develop algorithms to interpret grammar specifications and generate inputs accordingly.
• Explore different paths and options within the grammar to maximise the branch coverage.
• Test the generated inputs on the target program to assess its branch coverage.
• Implement mechanisms to adjust the generation process to hit or exceed the input and code
coverage threshold.
Input Specifications
Your program should take 3 command-line arguments:
1. The path to a Python script
2. The path to a single Python (.py) script containing the grammar specifications using the syntax
taught in the lectures and tutorials; the grammar will be stored as the variable ’grammar’
3. The number of strings your program should generate for the test suite
It should be called using the following command:
python grammar_fuzzer.py
Page 5
Output Specifications
The program should generate structured inputs based on the grammar specifications provided and
write them to an output ‘.in‘ file with the same name as the program itself. The output file should
contain the specified number of strings each on a new line, where each string represents a test input.
The generated inputs should cover various paths and options within the grammar, aiming to hit or
exceed the branch coverage threshold defined for the target program.
For example, if the desired number of strings is 100:
1 input_1
2 input_2
3 ...
4 input_100
For example, if the program being run was
my_program.py
The file that the inputs would be written to would be
my_program.in
Make sure that the generated inputs cover as many grammar rules and options as possible to effectively
test the target program and meet the input and code coverage threshold.
Getting Started
• Review Tutorials and Lectures: Begin by reviewing the tutorials and lectures. Remember
that everything you need for each component has already been covered in this unit.
• Understand the Fundamentals: Go through the revision slides on Ed and make sure that
you understand all of the content covered so far.
• Ask Questions: If you have any questions or uncertainties about the material covered, don’t
hesitate to ask on Ed for clarification and a TA will get back to you shortly.
Frequently Asked Questions
• Hard coding will result in a 0 for all tasks.
• No external libraries (i.e. those installed using pip or another package manager) may be
used - this is a limitation of Edstem.
• You have unlimited attempts before the deadline.
• There are public, private and hidden test cases for all tasks.
• Test cases will gradually be released over the coming days, and you should check Ed for
announcements.
• You may reuse their code from Quiz 1 and any other task from this unit.
• All code, even your own, must be referenced as per the university’s policy.
• You may structure your program as you wish as long as it is written in Python and gets
called using the described commands.
Page 6