"Downloaded and installed/compile tacoco repo"

*.json

*.exec

/bin

*.class

*swp

/target

# Mobile Tools for Java (J2ME)

.mtj.tmp/

# Package Files #

*.jar

*.war

*.ear

# virtual machine crash logs, see http://www.java.com/en/download/help/error_hotspot.xml

hs_err_pid*

*.pydevproject

.metadata

.gradle

bin/

tmp/

*.tmp

*.bak

*.swp

*~.nib

local.properties

.settings/

.loadpath

# Eclipse Core

.project

# External tool builders

.externalToolBuilders/

# Locally stored "Eclipse launch configurations"

*.launch

# CDT-specific

.cproject

# JDT-specific (Eclipse Java Development Tools)

.classpath

# Java annotation processor (APT)

.factorypath

# PDT-specific

.buildpath

# sbteclipse plugin

.target

# TeXlipse plugin

.texlipse

cp.txt

language: java

jdk: 

- oraclejdk8

- oraclejdk7

before_install:

- git clone https://github.com/inf295uci-2015/primitive-hamcrest.git  

- cd primitive-hamcrest  

- mvn install -Dmaven.compiler.source="1.7"  -Dmaven.compiler.target="1.7"

- cd ..

after_success:

- mvn clean test jacoco:report coveralls:report

 No newline at end of file

# tacoco

[![Build Status](https://travis-ci.org/inf295uci-2015/tacoco.svg?branch=master)](https://travis-ci.org/inf295uci-2015/tacoco)

[![Coverage Status](https://coveralls.io/repos/inf295uci-2015/tacoco/badge.svg?branch=master)](https://coveralls.io/r/inf295uci-2015/tacoco?branch=master)

## Compiling Tacoco

1. Install [Primitive Hamcrest (https://github.com/inf295uci-2015/primitive-hamcrest)](https://github.com/inf295uci-2015/primitive-hamcrest) in your local repository (check instruction in given in primitive-hamcrest's Readme.md).

2. Run `mvn comile` as a sanity check to make sure that there are no compile-time errors.

3. Run `mvn test` to make sure that tacoco is working against its own test-cases.

## Analyzing with Tacoco

* Get fresh tacoco

~~~

  git clone https://github.com/spideruci/tacoco

  mvn compile

~~~

* Run tacoco 

~~~

cd /to/your/project/root

mvn test

mvn dependency:build-classpath -Dmdep.outputFile=cp.txt

export CLASSPATH=`cat cp.txt`:$CLASSPATH

export CLASSPATH={your project absolute path}/target/test-classes:{your project absolute path}/target/classes:$CLASSPATH

cd /to/tacoco/project/root

mvn dependency:build-classpath -Dmdep.outputFile=cp.txt

export CLASSPATH=`cat cp.txt`:$CLASSPATH

export CLASSPATH={tacoco project absolute path}/target/test-classes:{tacoco project absolute path}/target/classes:$CLASSPATH

mvn dependency:copy-dependencies -DoutputDirectory=lib

java -javaagent:lib/org.jacoco.agent-0.7.4.201502262128-runtime.jar=destfile=jacoco.exec,dumponexit=false org.spideruci.tacoco.TacocoRunner {your project absolute path}/target/test-classes

~~~

* Alternatively use the `export-sut-cp` to Run tacoco

~~~

## make sure that you run `mvn test` on your project

cd /to/your/project/root

mvn test

## switch to tacoco and run export-sut-cp

cd /to/tacoco/project/root

cd scripts

chmod +x export-sut-cp # you need to do this just once.

./export-sut-cp absolute/path/of/your/project/root absolute/path/of/tacoco/root

## this should create a jacoco.exec file in the `absolute/path/of/tacoco/root`

~~~

## Running ExecAnalyzer utility to parse the `jacoco.exec` file

### Help Menu

```

tacoco$ mvn -q exec:java -Panalyzer -Dtacoco.help

Tacoco: Exec-file Analyzer

usage: mvn exec:java -q -Panalyzer [arguments] 

Arguments:

-Dtacoco.sut=<dir>                  (Required) Absolute-path of system-

                                    under-test's root.

-Dtacoco.exec=<*.exec>              (Required) Absolute-path of input exec

                                    binary.

-Dtacoco.json=<*.json>              (Default: STDOUT) Absolute-path of per-test

                                    coverage output.

-Dtacoco.fmt=<LOOSE|COMPACT|DENSE>  (Default: DENSE) Compression format of

                                    coverage data.

-Dtacoco.pp                         Pretty prints coverage data to json file.

-Dtacoco.help                       Prints this message and exits (with 0).

```

### Step-wise instructions

1. Compile the ExecAnalyzer as stated above.

2. Use the `mvn` command on the command line to execute the ExecAnalyzer as shown above in the help menu, e.g.: `mvn exec:java -q -Panalyzer -Dtacoco.sut=/home/vijay/misc_Programming/pmd/pmd-java -Dtacoco.json=pmd-java-compact.json -Dtacoco.exec=jacoco.exec -Dtacoco.pp -Dtacoco.fmt=COMPACT`

    * `-Dtacoco.pp` and `-Dtacoco.help` are treated as flags.

    * `-Dtacoco.json=[*.json]` and `Dtacoco.fmt=[LOOSE|COMPACT|DENSE]`. Not specifying those options will result in the selection of default options for each of those arguments.

    * The default options are:

        * `-Dtacoco.json=[*.json]` -- **`System.out` i.e. Standard-Out**

        * `-Dtacoco.fmt=[LOOSE|COMPACT|DENSE]` -- **`DENSE`**

    * `-Dtacoco.pp`, being a flag, is also optional. Not providing it means that you do not want pretty-printed coverage-data.

3. NOTE: Coverage Compression Formats and Pretty-printing are two different things. Pretty-printing simply ensures that the Json output is printed in a non-minified manner. Continue reading to learn more about coverage compression formats.

### Notes on Line-coverage Compression Formats

You have 3 choices for `-Dtacoco.fmt=`: **`LOOSE`, `COMPACT`, `DENSE`**

#### LOOSE Format

- This is a space-inefficient formatting of the coverage information.

#### COMPACT Format

- Bit-based encoding for each line-level coverage information.

- Line-level coverage information encoded to single 32-bit int.

- Each (source) line has the following two counters:

  - Bytecode Instruction Coverage Counter (number of bytecode instructions covered and missed)

  - Branch Coverage Counter (number of branches covered and missed)

- Compression Scheme is as follows:

  - (starting from the most significant bits)

  - First 8 bits encode number of bytecode instructions covered,

  - Next 8 bits encode number of bytecode instructions missed,

  - Next 8 bits encode number of branches covered,

  - Next 8 bits encode number of branches missed.

- Assuming: each line contains max 255 bytecode instructions or branches.

#### DENSE Format

- Only the statuses of the lines i.e. EMPTY, NOT_COVERED, PARTLY_COVERED, FULLY_COVERED are encoded in DENSE formatting.

- 2 bits are used per line for each line status: EMPTY=00, NOT_COVERED=01, PARTLY_COVERED=11, FULLY_COVERED=10.

- Instruction and Branch counters are **disregarded** in DENSE formatting.

- DENSE formatting encodes 16 (at most) line-statuses into a single 32-bit integer.

- DENSE formatting is implemented in LinesStatusCoder.

## Running CoverageJsonReader

```tex

tacoco$ mvn -q exec:java -Preader -Dtacoco.help

Tacoco: Coverage Json-file Reader

usage: mvn exec:java -q -Preader [arguments] 

Arguments:

-Dtacoco.json=<*.json>  (Required) Absolute-path of per-test coverage file.

-Dtacoco.out=<*.json>   Absolute-path of per-sourcefile coverage matrix.

-Dtacoco.pp             Pretty prints coverage data to json file.

-Dtacoco.help           Prints this message and exits (with 0).

```

1. Use the following maven command on the command line to execute the CoverageJsonReader: `mvn exec:java -Preader -Dtacoco.json="/path/to/your/json/coverage-data-file.json"`

## Space Optimized Coverage Matrix

### Per source file Coverage matrix

- The coverage matrix for a project is split into smaller coverage matrices for each source file.

- The statements in this matrix are localized to only the statements in the sourcefile in question.

- We maintain a list of tests for which the test-statement matrix is built.

- Tests that do not execute a single statement in the sourcefile are considered IRRELEVANT and are not a part of the test-statement matrix.

- Check `org.spideruci.tacoco.coverage.CoverageMatrix2` and `org.spideruci.tacoco.coverage.SourceSpecificCoverageMatrix` to study the models for per-source coverage.

- More to come ...

  + converting the boolean arrays into bit arrays/vectors

  + (optional) including an index to the test-case names. currently only the test-case-ids are listed in the per-source-coverage-matrices.

<!-- START doctoc generated TOC please keep comment here to allow auto update -->

<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->

**Table of Contents**  *generated with [DocToc](https://github.com/thlorenz/doctoc)*

- [Developer Notes](#developer-notes)

  - [Old Suggested POM Changes](#old-suggested-pom-changes)

  - [Testing](#testing)

    - [Exploratory Testing Session for `tacoco/TacocoRunner`](#exploratory-testing-session-for-tacocotacocorunner)

      - [Preparing SourceSurfer (system-under-analysis)](#preparing-sourcesurfer-system-under-analysis)

      - [Preparing tacoco](#preparing-tacoco)

      - [Export SourceSurfer dependencies and classes to $CLASSPATH](#export-sourcesurfer-dependencies-and-classes-to-classpath)

      - [Export tacoco dependencies and classes to $CLASSPATH](#export-tacoco-dependencies-and-classes-to-classpath)

      - [Run SourceSurfer's Test Classes from within tacoco](#run-sourcesurfers-test-classes-from-within-tacoco)

<!-- END doctoc generated TOC please keep comment here to allow auto update -->

# Developer Notes

## Old Suggested POM Changes

This was useful back when we had to manually change the pom.xml of the system under test. This stopped being necessary ever since the Pull Request #11. This note is purely for archival purposes, in case we need to learn something from the past.

~~~xml

  <build>

    <plugins>

    ...

      <plugin>

        <groupId>org.apache.maven.plugins</groupId>

        <artifactId>maven-surefire-plugin</artifactId>

        <version>2.18.1</version>

        <configuration>

          <properties>

            <property>

              <name>listener</name>

              <value>org.spideruci.tacoco.TacocoListener</value>

            </property>

          </properties>

          <additionalClasspathElements>

            <additionalClasspathElement>tacoco/target/tacoco-0.1.jar</additionalClasspathElement>

          </additionalClasspathElements>

        </configuration>

      </plugin>

      <plugin>

        <groupId>org.jacoco</groupId>

        <artifactId>jacoco-maven-plugin</artifactId>

        <version>0.7.4.201502262128</version>

        <configuration>

          <destFile>jacoco.exec</destFile>

          <append>true</append>

        </configuration>

        <executions>

          <execution>

            <id>jacoco-initialize</id>

            <phase>initialize</phase>

            <goals>

              <goal>prepare-agent</goal>

            </goals>

          </execution>

          <execution>

            <id>jacoco-report</id>

            <phase>verify</phase>

            <goals>

              <goal>report</goal>

            </goals>

          </execution>

        </executions>

      </plugin>

    ...

~~~

## Testing

### Exploratory Testing Session for `tacoco/TacocoRunner`

**Ref.: Pull-Request #11**  

**System-under-analysis: SourceSurfer**  

**Tester: @vijaykrishna**  

#### Preparing SourceSurfer (system-under-analysis)

- Get a fresh copy of SourceSurfer (system under analysis) ... `git clone https://github.com/inf295uci-2015/SourceSurfer.git`

- Enter the system's directory ... `cd SourceSurfer/`

- Compile the sources of the system ... `mvn compile`

    - Build was successful.

    - `Total time: 2.760 s`; `Finished at: 2015-05-26T16:23:30-08:00`; `Final Memory: 11M/110M`

- Test the compiled system ... `mvn test`

    - Tests were successful: `Tests run: 25, Failures: 0, Errors: 0, Skipped: 0`

    - `Total time: 3.464 s`; `Finished at: 2015-05-26T16:32:25-08:00`; `Final Memory: 13M/110M`

- The system-under-analysis is now ready.

#### Preparing tacoco

- (note: i already had primitive-hamcrest installed on my local machine. check README.md for more details on how to install primitive-hamcrest.)

- switch to home: `cd`

- Get a fresh copy of tacoco: `git clone https://github.com/inf295uci-2015/tacoco`

- Enter tacoco's directory: `cd tacoco`

    - I also had to pull the latest version of the `TacocoRunner` branch: `git pull origin TacocoRunner`. This step will not be necessary if pull request #11 has been approved, merged and closed.

- Compile the sources of tacoco: `mvn compile`

    + Build was successful.

    + `Total time: 2.285 s`; `Finished at: 2015-05-26T16:45:59-08:00`, `Final Memory: 16M/109M`.

- Test the tacoco build: `mvn test`

    + Tests were successful: `Tests run: 4108, Failures: 0, Errors: 0, Skipped: 0`.

    + `Total time: 5.274 s`; `Finished at: 2015-05-26T16:47:27-08:00`; `Final Memory: 10M/174M`.

- tacoco, with the `TacocoRunner` is now ready.

#### Export SourceSurfer dependencies and classes to $CLASSPATH

- Switch to ~/SourceSurfer

- Spit out the paths to SourceSurfer's dependencies with maven dependency into a text file: `SourceSurfer$ mvn dependency:build-classpath -Dmdep.outputFile=cp.txt`

    - Build was successful

    - The contents of cp.txt as a result of the build:

```tex

/home/vijay/.m2/repository/com/esotericsoftware/yamlbeans/yamlbeans/1.

09/yamlbeans-1.09.jar:/home/vijay/.m2/repository/org/mockito/mockito-core/1.10.

19/mockito-core-1.10.19.jar:/home/vijay/.m2/repository/org/hamcrest/hamcrest-

core/1.1/hamcrest-core-1.1.jar:/home/vijay/.

m2/repository/org/objenesis/objenesis/2.1/objenesis-2.1.jar:/home/vijay/.

m2/repository/com/cedarsoftware/json-io/3.0.2/json-io-3.0.2.jar:/home/vijay/.

m2/repository/commons-cli/commons-cli/1.2/commons-cli-1.2.jar:/home/vijay/.

m2/repository/junit/junit/4.12/junit-4.12.jar:/home/vijay/.

m2/repository/org/hamcrest/hamcrest-library/1.3/hamcrest-library-1.3.

jar:/home/vijay/.m2/repository/org/hamcrest/hamcrest-junit/2.0.0.0/hamcrest-

junit-2.0.0.0.jar:/home/vijay/.m2/repository/org/hamcrest/java-hamcrest/2.0.0.

0/java-hamcrest-2.0.0.0.jar

```

- Verify $CLASSPATH: `echo $CLASSPATH`. $CLASSPATH was empty.

- Export contents of cp.txt to $CLASSPATH: `export CLASSPATH=``cat cp.txt``:$CLASSPATH`

- Verify $CLASSPATH: `echo $CLASSPATH`.

    - Result -- Success! The paths to SourceSurfer's dependencies were added to the classpath, as shown below:

```tex

/home/vijay/.m2/repository/com/esotericsoftware/yamlbeans/yamlbeans/1.

09/yamlbeans-1.09.jar:/home/vijay/.m2/repository/org/mockito/mockito-core/1.10.

19/mockito-core-1.10.19.jar:/home/vijay/.m2/repository/org/hamcrest/hamcrest-

core/1.1/hamcrest-core-1.1.jar:/home/vijay/.

m2/repository/org/objenesis/objenesis/2.1/objenesis-2.1.jar:/home/vijay/.

m2/repository/com/cedarsoftware/json-io/3.0.2/json-io-3.0.2.jar:/home/vijay/.

m2/repository/commons-cli/commons-cli/1.2/commons-cli-1.2.jar:/home/vijay/.

m2/repository/junit/junit/4.12/junit-4.12.jar:/home/vijay/.

m2/repository/org/hamcrest/hamcrest-library/1.3/hamcrest-library-1.3.

jar:/home/vijay/.m2/repository/org/hamcrest/hamcrest-junit/2.0.0.0/hamcrest-

junit-2.0.0.0.jar:/home/vijay/.m2/repository/org/hamcrest/java-hamcrest/2.0.0.

0/java-hamcrest-2.0.0.0.jar:

```

- Export paths of the classfiles in the target directory (`./target/classes` and `./target/test-classes`) to $CLASSPATH: `export CLASSPATH=./target/test-classes:./target/classes:$CLASSPATH`

- Verify $CLASSPATH: `echo $CLASSPATH`. 

    - Result: Failed, because it added the related paths to the classpath. need to export the absolute paths of the target classes.

- Export **absolute** paths of the classfiles in the target directory to $CLASSPATH: `export CLASSPATH=/home/vijay/SourceSurfer/target/test-classes:/home/vijay/SourceSurfer/target/classes:$CLASSPATH`

- Verify $CLASSPATH: `echo $CLASSPATH`

    + Result: Success! The following string was added in front of $CLASSPATH variable: `/home/vijay/phd-open-source/SourceSurfer/target/test-classes:/home/vijay/phd-open-source/SourceSurfer/target/classes`

- The paths to the dependencies and the target classes of SourceSurfer were successfully exported to the $CLASSPATH variable.

#### Export tacoco dependencies and classes to $CLASSPATH

- Switch to tacoco's directory: `cd ~/tacoco`

- Spit out the paths of tacoco's dependencies into cp.txt `mvn dependency:build-classpath -Dmdep.outputFile=cp.txt`

    - Build was successful, and a mountain of dependencies are now available in cp.txt inside the tacoco directory. (the skipping the contents here, because there just too many dependencies).

- Verify $CLASSPATH to ensure that we have SourceSurfer's dependencies and classes on the $CLASPATH: `echo $CLASSPATH`. 

    - Result: $CLASSPATH has SourceSurfer's dependencies and classes.

    - Note: make sure that you do not open a new termnial session, otherwise $CLASSPATH will be empty --- the $CLASSPATH variable is local for each terminal session.

- Export cp.txt to $CLASSPATH: `export CLASSPATH=``cat cp.txt``:$CLASSPATH`.

    - Result: $CLASSPATH now has the dependencies of tacoco, in addition to the paths of SourceSurfer's dependencies and target classes.

- Export tacoco's target classes and test-classes to $CLASSPATH: `export CLASSPATH=/home/vijay/tacoco/target/test-classes:/home/vijay/tacoco/target/classes:$CLASSPATH`

- Verify $CLASSPATH: `echo $CLASSPATH`

    - Success! -- The paths for the classes and target-classes were added in front of the $CLASSPATH.

- Copy tacoco's dependencies (jar files) into a folder called lib: `mvn dependency:copy-dependencies -DoutputDirectory=lib`.

    + Result: All of tacoco's dependent jars are now in a newly created folder called `lib` under tacoco (i.e. `~/tacoco/lib`).

#### Run SourceSurfer's Test Classes from within tacoco

- Before running the following command, there was no jacoco.exec file in tacoco's project directory. I expect there to be one, when i run the following command.

- Run: `java -javaagent:lib/org.jacoco.agent-0.7.4.201502262128-runtime.jar=destfile=jacoco.exec,dumponexit=false org.spideruci.tacoco.TacocoRunner /home/vijay/SourceSurfer/target/test-classes`

- Results:

    - the tacocolistener seems to have been invoked as i see the following familiar messages on the terminal: "Test case finished. Going to sleep for 10 ms. Done sleeping."

    - there was a jacoco.exec file created after the execution of the above command: `-rw-rw-r-- 1 vijay vijay 315822 May 26 17:39 jacoco.exec`.

    - Successfully executed the `ExecAnalyzer` to spit out the json file from jacoco.exec (i had to change the mainClass to ExecAnalyzer in pom.xml): `mvn exec:java -Dexec.args="/home/vijay/SourceSurfer /home/vijay/tacoco/jacoco.exec dense.json DENSE true"`

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

	xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">

	<modelVersion>4.0.0</modelVersion>

	<groupId>org.spideruci.tacoco</groupId>

	<artifactId>tacoco</artifactId>

	<version>0.1</version>

	<packaging>jar</packaging>

	<properties>

		<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>

		<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>

	</properties>

	<name>tacoco the per testcase junit runner</name>

	<url></url>

	<dependencies>

		<dependency>

			<groupId>org.jacoco</groupId>

			<artifactId>jacoco-maven-plugin</artifactId>

			<version>0.7.4.201502262128</version>

		</dependency>

		<dependency>

			<groupId>junit</groupId>

			<artifactId>junit</artifactId>

			<version>4.12</version>

		</dependency>

		<dependency>

			<groupId>com.google.code.gson</groupId>

			<artifactId>gson</artifactId>

			<version>2.3.1</version>

		</dependency>

		<dependency>

			<groupId>org.mockito</groupId>

			<artifactId>mockito-core</artifactId>

			<version>2.0.8-beta</version>

			<scope>test</scope>

		</dependency>

		<dependency>

			<groupId>org.hamcrest</groupId>

			<artifactId>java-hamcrest</artifactId>

			<version>2.0.0.0</version>

			<scope>test</scope>

		</dependency>

		<dependency>

			<groupId>org.spideruci.hamcrest</groupId>

			<artifactId>primitive-hamcrest</artifactId>

			<version>0.0.1-SNAPSHOT</version>

		</dependency>

	</dependencies>

	<build>

		<plugins>

			<plugin>

				<groupId>org.apache.maven.plugins</groupId>

				<artifactId>maven-compiler-plugin</artifactId>

				<version>3.1</version>

				<configuration>

					<source>1.7</source>

					<target>1.7</target>

				</configuration>

			</plugin>

			<plugin>

				<groupId>org.eluder.coveralls</groupId>

				<artifactId>coveralls-maven-plugin</artifactId>

				<version>3.1.0</version>

			</plugin>

			<plugin>

				<groupId>org.jacoco</groupId>

				<artifactId>jacoco-maven-plugin</artifactId>

				<version>0.7.4.201502262128</version>

				<executions>

					<execution>

						<id>prepare-agent</id>

						<goals>

							<goal>prepare-agent</goal>

						</goals>

					</execution>

				</executions>

			</plugin>

		</plugins>

	</build>

	<profiles>

		<profile>

			<id>analyzer</id>

			<build>

				<plugins>

					<plugin>

						<groupId>org.codehaus.mojo</groupId>

						<artifactId>exec-maven-plugin</artifactId>

						<version>1.4.0</version>

						<configuration>

							<mainClass>org.spideruci.tacoco.reporting.ExecAnalyzer</mainClass>

						</configuration>

					</plugin>

				</plugins>

			</build>

		</profile>

		<profile>

			<id>runner</id>

			<build>

				<plugins>

					<plugin>

						<groupId>org.codehaus.mojo</groupId>

						<artifactId>exec-maven-plugin</artifactId>

						<version>1.4.0</version>

						<configuration>

							<mainClass>org.spideruci.tacoco.TacocoRunner</mainClass>

						</configuration>

					</plugin>

				</plugins>

			</build>

		</profile>

		<profile>

			<id>reader</id>

			<build>

				<plugins>

					<plugin>

						<groupId>org.codehaus.mojo</groupId>

						<artifactId>exec-maven-plugin</artifactId>

						<version>1.4.0</version>

						<configuration>

							<mainClass>org.spideruci.tacoco.reporting.CoverageJsonReader</mainClass>

						</configuration>

					</plugin>

				</plugins>

			</build>

		</profile>

	</profiles>

</project>