Google Summer of Code 2018 Work Product Submission



coala

Rahul Jha

Rahul is currently a junior, studying Electronics Engineering at Zakir Hussain College of Engineering and Technology, Aligarh Muslim University. He loves the ideology behind Free and Open Source Software and reads about the history of this revolutionary movement in his leisure time.


Patches Tarball


SHA-256:

a722943c4a4191f1e624d4dc5f36cc3da4f160a91544d26bc55837ee4ba4e5dc

Bonding

Phase 1

Phase 2

Phase 3


Links to commits and repositories I've worked on:

Repository Link to Commit/s Description
p  projects View

Add automatic whitelist generator for Vulture

v  vulture-whitelist-generators View

Package code in modules and improve documentation

v  vulture-whitelist-generators View

Ignore dist/; minor improvements to docs

v  vulture-whitelist-generators View

Remove nargs=1

v  vulture-whitelist-generators View

Minor Cosmetics

v  vulture-whitelist-generators View

Rename genereators/qt_utils.py to generators/qt.py

v  vulture-whitelist-generators View

Rename generators to vulture_whitelist

v  vulture-whitelist-generators View

Use QtWhitelistCreator as a Creator object

v  vulture-whitelist-generators View

Add tests and configure test suite

v  vulture-whitelist-generators View

Add travis and coveralls

v  vulture-whitelist-generators View

Add sip as a dependency

v  vulture-whitelist-generators View

Install sip in travis

v  vulture-whitelist-generators View

Sort before writing xml

v  vulture-whitelist-generators View

Install lxml when using local repository.

v  vulture-whitelist-generators View

setup.py: test requires vulture, not sip

v  vulture-whitelist-generators View

README.rst: Minor nitpicks and cosmetics

v  vulture-whitelist-generators View

setup.py: mention lxml only in install_requires

v  vulture-whitelist-generators View

setup.py: Remove pytest runner and import version correctly

v  vulture-whitelist-generators View

tests/init.py: TESTS and DIR are same

v  vulture-whitelist-generators View

Use Creator object to register new creators

v  vulture-whitelist-generators View

tests/test_with_vulture.py: Use pytest.fixture

v  vulture-whitelist-generators View

tests/test_with_vulture.py: Use textwrap.dedent

v  vulture-whitelist-generators View

Use absolute imports everywhere

v  vulture-whitelist-generators View

Use python’s logging module.

v  vulture-whitelist-generators View

Define version in vulture_whitelist/main.py

v  vulture-whitelist-generators View

Bump up PyQt versions, update features

v  vulture-whitelist-generators View

Use pytest’s tmpdir fixture

v  vulture-whitelist-generators View

Single source package version

v  vulture View

README.rst: Add examples to setup (#114)

v  vulture View

Update tests to only pass supported arguments to literal_eval (#118)

v  vulture View

Fix PEP8 issues (#120)

v  vulture View

Update TODO file (#122)

v  vulture View

Add coveralls support to track code coverage (#124)

v  vulture View

Document exit codes (#123)

v  vulture View

Increase test coverage (#126)

v  vulture View

Add coveralls badge to README (#127)

v  vulture View

Use argparse instead of optparse (#119)

v  vulture View

Test file with non utf-8 encoding (#121)

v  vulture View

Skip async tests correctly (#128)

v  vulture View

tests/test_errors.py: Use pytest’s tmpdir fixture (#131)

v  vulture View

Use absolute imports (#132)

v  vulture View

Single-source package version string (#134)

v  vulture View

Whitelists: use non-dynamic objects directly instead of mocking them. (#137)

v  vulture View

Add –make-whitelist flag (#135).

v  vulture View

Add –ignore-names flag (#139).

v  vulture View

Add Item.get_whitelist_string. (#144)

v  vulture-analysis View

Add analysis report for localstack

v  vulture-analysis View

Pre-format logs

v  vulture-analysis View

Update localstack_analysis.md

v  vulture-analysis View

Make report conform to the given template

v  vulture-analysis View

Update localstack_analysis.md

v  vulture-analysis View

create analysis report for system design primer

v  vulture-analysis View

Organise reports into folders.

v  vulture-analysis View

add header for localstack

v  vulture-analysis View

Add report for relational-networks

v  vulture-analysis View

Add footer

v  vulture-analysis View

Add report for qutebrowser (#4)

v  vulture-analysis View

Add report for Sharppy (#5)


Improving whitelisting infrastructure for Vulture

Work Done

  1. Created a command line tool which takes in sip files as input and spits a whitelist with all the virtual functions defined in the sip files. This tool was further used to create a whitelist for PyQt which can hereby be used with any project using PyQt. The tool is now completely tested and documented.

  2. Created --make-whitelist flag to enable reporting of the results in a format which can be directly used as a whitelist. This significantly eased the process of creating and maintaing a whitelist for a project.

  3. Implemented an ignoring mechanism for Vulture (--ignore-names and --ignore-decorators) - thereby enabling users to omit the result(s) which match with the given argument(s) (a glob pattern). This is particularly useful for projects using Flask where --ignore-decorators "@app.route" can be used to ignore all functions with the @app.route decorator (The functions exposed via the public API which aren’t used internally).

  4. Increased test coverage of Vulture from already excellent 95% to 100%. Also, ensured support for Python3.7 in CI and started testing Vulture with Python3.8-dev.

  5. Switch to argparse instead of using optparse in Vulture.

  6. We also did an experimental integration with coverage so that we could automatically remove the false positives. However, this scheme didn’t quite fit the workflow involved when working with Vulture, so we dropped the idea.

  7. Along with documenting the new features in Vulture, the already exisitng docs were also added upon. Most significantly, we tabulated the exit codes for Vulture and mentioned a few examples with their respective command and output in the project’s README section.

Challenges

The biggest challenge for me was to ensure that the workflow involved for using Vulture stays simple and that we don’t overengineer it. Thankfully, my mentors could foresee this and helped me throughout.