Google Summer of Code 2018 Work Product Submission



coala

Ankit Joshi

A senior student at National Institute Of Technology, Kurukshetra major in Information Technology. For GSoC, I worked on coala-pyflakes and pyflakes. The project was to integrate pyflakes-enhanced AST into coala as a metabear which was then used to develop various plugins. The second part of the project was to write plugin bears using the developed PyFlakesASTBear.


Patches Tarball


SHA-256:

cf0c8e3daded8b61384a7ebf7d5ce17b535ce165f9e4694c986f1bf9e4ab9e68

Bonding

Phase 1

Phase 2

Phase 3


Links to commits and repositories I've worked on:

Repository Link to Commit/s Description
c  coala-pyflakes View

Initial commit

c  coala-pyflakes View

PyFlakesASTBear.py: Add PyFlakesASTBear

c  coala-pyflakes View

coafile: Add .coafile

c  coala-pyflakes View

PyFlakesASTBearTest.py: Add PyFlakesASTBearTest

c  coala-pyflakes View

pyflakesbears: Package renamed

c  coala-pyflakes View

NoFutureImportBear.py: Add NoFutureImportBear

c  coala-pyflakes View

.gitignore: Remove test cache

c  coala-pyflakes View

coala-pyflakes: Add requirements

c  coala-pyflakes View

coala-pyflakes: Add circleci

c  coala-pyflakes View

requirements.txt: Add coala-bears

c  coala-pyflakes View

requirements.txt: Update coala version

c  coala-pyflakes View

pyflakes_bears: Rename pyflakesbears

c  coala-pyflakes View

.config.yml: Run coala

c  coala-pyflakes View

NoFutureImportBear.py: Add NoFutureImportBear

c  coala-pyflakes View

Integrate moban

c  coala-pyflakes View

pyflakes_generic_plugins: Add generic plugins

c  coala-pyflakes View

Sync moban

c  coala-pyflakes View

coala-pyflakes: Sync moban

c  coala-pyflakes View

coala-pyflakes: Sync moban

c  coala-pyflakes View

docs: Add documentation

c  coala-pyflakes View

.ci: Restructure ci in repo

c  coala-pyflakes View

Sync moban

c  coala-pyflakes View

Sync moban

c  coala-pyflakes View

PEP8DoctestNamingBear.py: Add bear

c  coala-pyflakes View

PEP8DoctestNamingBear: Rename to PEP8NamingBear

c  coala-pyflakes View

PyFlakesASTBear.py: Remove ARGUMENT handler

c  coala-pyflakes View

PyFlakesASTBear.py: Track scope nodes

c  coala-pyflakes View

coala-pyflakes: Sync moban

c  coala-pyflakes View

PyFlakesASTBear.py: Override checker handler

c  coala-pyflakes View

setup.py: Add entrypoint for coala

c  coala-pyflakes View

coala-pyflakes: Add LICENSE

c  coala-pyflakes View

coala-pyflakes: Add CONTRIBUTING.rst

c  coala-pyflakes View

coala-pyflakes: Update LICENSE

c  coala-pyflakes View

coala-pyflakes: Add README.rst

c  coala-pyflakes View

.coafile: Add bandit bear

c  coala-pyflakes View

.coafile: Add PyCommentedCodeBear

c  coala-pyflakes View

.coafile: Add PyImportSortBear

c  coala-pyflakes View

.coafile: Add PyDocStyleBear

c  coala-pyflakes View

coala-pyflakes: Fix import order

c  coala-pyflakes View

PEP8NamingBear: Set col offset 0 to 1

c  coala-pyflakes View

.gitlab-ci.yml: Use moban template

c  coala-pyflakes View

NoFutureImportBear: Sort FutureImportation

c  coala-pyflakes View

coala-pyflakes: Sync moban

c  coala-pyflakes View

PyFlakesASTBear: Allow sorting of nodes

c  coala-pyflakes View

.ci: Remove travis and appveyor

c  coala-pyflakes View

.moban.yaml: Point setup.py to gitlab repo

c  coala-pyflakes View

pyflakes_bears: Change LICENSE to MIT

c  coala-pyflakes View

BuiltinsBear.py: Add BuiltinsBear

c  coala-pyflakes View

coala-pyflakes: Sync moban

c  coala-pyflakes View

requirements.txt: Use bears master

p  pyflakes View

Handle subtrees as input to checker

p  pyflakes View

checker.py: Handle UnboundLocal error for builtins

p  pyflakes View

checker.py: Check for invalid print syntax

c  coala View

Collectors.py: Sort bear list

c  coala View

Collectors.py: Document functions

c  coala View

ConfigurationGathering.py: Document functions

c  coala View

Sync moban

c  coala-bears View

PEP8Bear.py: Add asciinema

c  coala-bears View

tests/go/test_files: Change go test files

c  coala-bears View

RadonBear.py: Change bears settings

c  coala-bears View

Sync moban

c  coala-bears View

netlify.toml: Add netlify

c  coala-bears View

PyFlakesBear: Bump pyflakes to 2.0.0

c  cEPs View

Sync moban

c  cEPs View

.coafile: Set max_line_length

c  cEPs View

README.md: Add entry for cEP-0025

m  mobans View

test-requirements.txt: Ignore pip 18.0

m  mobans View

coala-setup.py.jj2: Fix zip_safe condition check

m  mobans View

coala-setup.py.jj2: Add zip_safe option

m  mobans View

coala-setup.py.jj2: Add yaml variable for docs_dir

m  mobans View

.moban.yml: Fix paths in copy

m  mobans View

assets/docs: Add docs-requirements.txt

m  mobans View

JetBrains.gitignore: Update with upstream

m  mobans View

coala-setup.py.jj2: Add multi package docs support

m  mobans View

appveyor.yml.jj2: Set path to python

c  coala-utils View

Sync moban

P  PyPrint View

Sync moban

P  PyPrint View

.coafile: Add NoFutureImport bear


Integrate Pyflakes-Enhanced-AST into coala

Work Done

There has not been an option to use the pyflakes-enhanced-AST based checker API to create custom linters. Thus, the full potential of enhanced AST isn’t utilized, a whole lot of rework is required to do the basic traversing and collection of important nodes. Pyflakes provides with a basic API that does the traversing. So, if a developer uses enhanced AST he just needs to work on the implementation of the new logic that his/her plugin provides and not about the fidelity of the basic node handlers.

Major tasks of the project:

  1. Create a separate repository coala-pyflakes to house all pyflakes based plugins.
  2. Create package pyflakes_bears to hold PyFlakesASTBear and all the coala bears developed using it.
  3. Create package pyflakes_generic_plugins that consists generic plugins which can be integrated with flake8 and other such similar wrappers.
  4. Design and develop PyFlakesASTBear that provides helper functions that can be used to traverse pyflakes AST of the input python module.
  5. Establish the proof of concept by creating NoFutureImportBear that detects and fixes any usage of __future__ imports in a python file.
  6. Add handlers for ARGS and ARGUMENTS and provide the functionality of partial code analysis in pyflakes.
  7. Create a PEP8NamingBear that detects all pep8 naming violations in a python file. Special handling is also done to detect violations in doctests which other similar plugins fail to perform.
  8. Create a BuiltinsBear that is capable of detecting redefinition of python builtins in a module.

Challenges

The challenge of the project was to implement the PyFlakesASTBear in such a way that a plugin developer need not worry about writing his own traversing logic and need to have much no more than the knowledge of PyFlakesASTBear nodes to write the bears. However, designing the logic first by writing cEP(coala enhancement proposal) for coala-pyflakes helped me a lot to understand the problem to a deep level and come out with a logic to tackle it.

Future prospects of the projects

The PyFlakesASTBear now allows bear developers to detect codesmells and anti-patterns in python programs allowing to create more bears and an equivalent generic plugin. coala-pyflakes can be used to safe house checks which are more strict hence can’t be directly incorporated in pyflakes. The bear framework can be further enhanced to include the functionality of executing flake8 plugin as is.