witter

所属分类:工具库
开发工具:kotlin
文件大小:0KB
下载次数:0
上传日期:2023-11-01 14:01:01
上 传 者sh-1993
说明:  介绍性编程算法的白盒测试库
(A Library for White-Box Testing of Introductory Programming Algorithms)

文件列表:
.idea/ (0, 2023-11-13)
.idea/gradle.xml (559, 2023-11-13)
.idea/inspectionProfiles/ (0, 2023-11-13)
.idea/inspectionProfiles/Project_Default.xml (276, 2023-11-13)
.idea/kotlinc.xml (176, 2023-11-13)
.idea/misc.xml (1239, 2023-11-13)
.idea/vcs.xml (167, 2023-11-13)
.idea/workspace.xml (22468, 2023-11-13)
build.gradle.kts (497, 2023-11-13)
gradle.properties (27, 2023-11-13)
gradle/ (0, 2023-11-13)
gradle/wrapper/ (0, 2023-11-13)
gradle/wrapper/gradle-wrapper.jar (60756, 2023-11-13)
gradle/wrapper/gradle-wrapper.properties (201, 2023-11-13)
gradlew (8070, 2023-11-13)
gradlew.bat (2763, 2023-11-13)
libs/ (0, 2023-11-13)
libs/strudel-0.8.1.jar (682112, 2023-11-13)
settings.gradle.kts (27, 2023-11-13)
src/ (0, 2023-11-13)
src/TSL.g4 (1140, 2023-11-13)
src/main/ (0, 2023-11-13)
src/main/java/ (0, 2023-11-13)
src/main/java/examples/ (0, 2023-11-13)
src/main/java/examples/paper/ (0, 2023-11-13)
src/main/java/examples/paper/binarysearch/ (0, 2023-11-13)
src/main/java/examples/paper/binarysearch/BinarySearch.java (582, 2023-11-13)
src/main/java/examples/paper/binarysearch/LinearSearch.java (249, 2023-11-13)
src/main/java/examples/paper/factorial/ (0, 2023-11-13)
src/main/java/examples/paper/factorial/IterativeFactorial.java (312, 2023-11-13)
src/main/java/examples/paper/factorial/RecursiveFactorial.java (301, 2023-11-13)
src/main/java/examples/paper/sorting/ (0, 2023-11-13)
src/main/java/examples/paper/sorting/InsertionSort.java (501, 2023-11-13)
src/main/java/examples/paper/sorting/SelectionSort.java (403, 2023-11-13)
src/main/kotlin/ (0, 2023-11-13)
src/main/kotlin/pt/ (0, 2023-11-13)
src/main/kotlin/pt/iscte/ (0, 2023-11-13)
src/main/kotlin/pt/iscte/witter/ (0, 2023-11-13)
... ...

# Witter **A Library for White-Box Testing of Introductory Programming Algorithms** [![ACM SPLASH'23 - Check out Witter's paper](https://img.shields.io/badge/ACM_SPLASH'23-Check_out_Witter's_paper-ebc034?logo=acm)](https://doi.org/10.1145/3622780.3623650) **Witter** is a software testing library that allows programming instructors to define white-box tests for Java source code. Witter analyzes the execution of a method against a reference solution, to verify that the code not only produces correct results but is also in accordance with a desired algorithm behaviour. [Installation](#installation) [Specifying reference solutions](#specifying-reference-solutions) [Testing arbitrary solutions](#testing-arbitrary-solutions) [Examples](#examples)

## Installation **Witter** is an experimental library, and as such is not yet available in build automation tools (Gradle, etc.) To use Witter in your project, first build its .jar file using Gradle's build task. The .jar file is generated under the project root in `/build/libs`. This file should be copied to your own project's `libs` folder, and then added as a dependency in your build automation tools. For example, in Gradle: ```kotlin dependencies { implementation(files("libs/witter-0.3.0.jar")) } ``` Note, of course, that the file name can change when updates for Witter are released, and should be changed in your dependency specification accordingly. You may additionally need to specify dependencies for the [Strudel](https://github.com/andre-santos-pt/strudel) and [ANTLR](https://www.antlr.org/) libraries.
## Specifying reference solutions One can define the test cases for a given exercise by writing a reference solution in a Java method, annotated with a header comment that defines the different test inputs and the metrics that should be measured during test execution. The content of the comments has to obey Witter’s Test Specification Language (TSL), whose syntax is similar to Java’s annotation syntax: ```java /* @Test(new int[] { 1, 2, 3, 4, 5 }) @Test(new int[] { 2, 4, 6 }) @CountLoopIterations @CountArrayReads @CheckSideEffects */ public static int sum(int[] a) { ... } ``` Witter currently supports the following runtime metrics. | **Metric** | **Annotation** | **Verification** | |--------------------|-------------------------------------|---------------------------------------------------------------------------------------------------| | Return values | @Test(_[...args]_) | Return value is equal to reference solution. Multiple annotations can be used. | | Side effects | @CheckSideEffects | Side effects on arguments (presence and absence) are the same to those of the reference solution. | | Loop iterations | @CountLoopIterations(_[threshold]_) | Total number of loop iterations matches the one of the reference solution. | | Array allocations | @CheckArrayAllocations | The array allocations match those of the reference solution (component types and lengths). | | Array reads | @CountArrayReads(_[threshold]_) | The number of array read accesses is the same as in the reference solution. | | Array writes | @CountArrayWrites(_[threshold]_) | The number of array write accesses is the same as in the reference solution. | | Object allocations | @CheckObjectAllocations | The number of object allocations and their types match those of the reference solution. | | Recursive calls | @CountRecursiveCalls(_[threshold]_) | The number of recursive calls matches the one of the reference solution. |
## Testing arbitrary solutions As Witter is designed for third-party integration, it provides a form of executing the tests programmatically. Tests are executed providing an annotated reference solution as described, and a solution that one wishes to assess: ```java Test test = new Test("ReferenceSolution.java"); List results = test.execute("Solution.java"); ``` The test results consist of a list of feedback items for each aspect defined in the test specification, holding the following information: - a flag indicating success or failure; - which kind of metric has failed (recall Table 1); - the location of code elements involved in the failed tests (e.g., procedure, parameters, loop structures); - a human-readable descriptive feedback message.
## Examples
Factorial (recursive) Reference solution with recursion: ```java /* @Test(5) @CountRecursiveCalls(1) */ static int factorial(int n) { if (n == 0) return 1; else return n * factorial (n - 1); } ``` Solution under testing (iterative, with a defect): ```java static int factorial(int n) { int f = 1; for (int i = 0; i <= n; i++) f *= i; // i starts at 0, f always 0 return f; } ``` Witter test results (black-box and white-box fail): ``` [fail] factorial(5) Expected result: 120 Found: 0 [fail] factorial(5) Expected recursive calls: 4 (± 1) Found: 0 ```

Binary search (iterative) Reference solution using binary search: ```java /* @Test(new int[] { 1, 2, 3, 4, 5, 6, 7 }, 1) @Test(new int[] { 1, 3, 7, 9, 11, 13, 17, 19 }, 18) @CountLoopIterations @CheckSideEffects */ static int binarySearch (int[] a, int e) { int l = 0; int r = a. length - 1; while (l <= r) { int m = l + (r - l) / 2; if (a[m] == e) return m; if (a[m] < e) l = m + 1; else r = m - 1; } return -1; } ``` Solution under testing (performing linear search): ```java static int binarySearch (int[] a, int e) { for (int i = 0; i < a. length ; i++) if (a[i] == e) return i; return -1; } ``` Witter test results (black-box pass, white-box fail): ``` [pass] search([1, 2, 3, 4, 5, 6, 7], 1) Expected result: 0 [fail] search([1, 2, 3, 4, 5, 6, 7], 1) Expected loop iterations: 3 Found: 1 [pass] search([1, 2, 3, 4, 5, 6, 7], 1) Expected side effects of a: [1, 2, 3, 4, 5, 6, 7] [pass] search([1, 2, 3, 4, 5, 6, 7], 1) Expected side effects of e: 1 [pass] search([1, 3, 7, 9, 11, 13, 17, 19], 18) Expected result: -1 [fail] search([1, 3, 7, 9, 11, 13, 17, 19], 18) Expected loop iterations: 4 Found: 8 [pass] search([1, 3, 7, 9, 11, 13, 17, 19], 18) Expected side effects of a: [1, 3, 7, 9, 11, 13, 17, 19] [pass] search([1, 3, 7, 9, 11, 13, 17, 19], 18) Expected side effects of e: 18 ```

Insertion sort (procedure) Reference solution performing insertion sort: ```java /* @Test(new int[] { 5, 4, 3, 2, 1 }) @CountArrayReads @CountArrayWrites @CheckSideEffects */ static void sort(int[] a) { for (int i = 1; i < a. length; i++) { for (int j = i; j > 0; j--) { if (a[j] >= a[j - 1]) break; int tmp = a[i]; a[i] = a[j]; a[j] = tmp; } } } ``` Solution under testing (performing selection sort): ```java static void sort(int[] a) { for (int i = 0; i < a. length - 1; i++) { int min = i; for (int j = i + 1; j < a. length ; j++) if (a[j] < a[min]) min = j; int tmp = a[i]; a[i] = a[min]; a[min] = tmp; } } ``` Witter test results (black-box pass, white-box fail): ``` [fail] sort([5, 4, 3, 2, 1]) Expected array reads: 40 Found: 28 [fail] sort([5, 4, 3, 2, 1]) Expected array writes: 20 Found: 8 [pass] sort([5, 4, 3, 2, 1]) Expected side effects of a: [5, 4, 3, 2, 1] ```

## See it in action The following is an example of how Witter could be integrated into an existing development system, using a simple GUI custom-made for example purposes. ![](witter-paddle-demo.gif)

近期下载者

相关文件


收藏者