Merge pull request #275 from bndr/next

Merge changes for version 0.4.11
This commit is contained in:
Alan Barzilay 2021-09-04 15:25:17 -03:00 committed by GitHub
commit dea950dd07
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
15 changed files with 1878 additions and 323 deletions

View File

@ -1 +0,0 @@
service_name: "travis-ci"

20
.github/workflows/flake8.yml vendored Normal file
View File

@ -0,0 +1,20 @@
name: flake8
on: pull_request
jobs:
flake8-lint:
runs-on: ubuntu-latest
name: Lint
steps:
- name: Check out source repository
uses: actions/checkout@v2
- name: Set up Python environment
uses: actions/setup-python@v2
with:
python-version: "3.9"
- name: flake8 Lint
uses: reviewdog/action-flake8@v3
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
reporter: github-pr-review

50
.github/workflows/tests.yml vendored Normal file
View File

@ -0,0 +1,50 @@
name: Tests and Codecov
on: pull_request
jobs:
run_tests:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
python-version: [3.6, 3.7, 3.8, 3.9, pypy-3.7]
steps:
- name: Checkout repository
uses: actions/checkout@v2
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install tox tox-gh-actions
- name: Test with tox
run: tox
coverage_report:
needs: run_tests
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v2
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install coverage docopt yarg requests
- name: Calculate coverage
run: coverage run --source=pipreqs -m unittest discover
- name: Create XML report
run: coverage xml
- name: Upload coverage to Codecov
uses: codecov/codecov-action@v2
with:
files: coverage.xml
fail_ci_if_error: true

View File

@ -1,30 +0,0 @@
# Config file for automatic testing at travis-ci.org
language: python
matrix:
include:
- python: 3.6
env: TOX_ENV=py36
- python: 3.5
env: TOX_ENV=py35
- python: 3.4
env: TOX_ENV=py34
- python: 2.7
env: TOX_ENV=py27
- python: pypy
env: TOX_ENV=pypy
- python: 3.6
env: TOX_ENV=flake8
# Use tox to run tests on Travis-CI to keep one unified method of running tests in any environment
install:
- pip install coverage coveralls tox
# Command to run tests, e.g. python setup.py test
script: tox -e $TOX_ENV
# Use after_success to get a single coveralls report
after_success:
- coverage run --source=pipreqs setup.py test
- coveralls

View File

@ -10,4 +10,5 @@ Development Lead
Contributors Contributors
------------ ------------
None yet. Why not be the first? * Jake Teo <mapattacker@gmail.com>
* Jerome Chan <cjerome94@gmail.com>

View File

@ -3,6 +3,11 @@
History History
------- -------
0.4.11 (2020-03-29)
--------------------
* Implement '--mode' (Jake Teo, Jerome Chan)
0.4.8 (2017-06-30) 0.4.8 (2017-06-30)
-------------------- --------------------

View File

@ -10,9 +10,8 @@
:target: https://pypi.python.org/pypi/pipreqs :target: https://pypi.python.org/pypi/pipreqs
.. image:: https://img.shields.io/coveralls/bndr/pipreqs.svg .. image:: https://codecov.io/gh/bndr/pipreqs/branch/master/graph/badge.svg?token=0rfPfUZEAX
:target: https://coveralls.io/r/bndr/pipreqs :target: https://codecov.io/gh/bndr/pipreqs
.. image:: https://img.shields.io/pypi/l/pipreqs.svg .. image:: https://img.shields.io/pypi/l/pipreqs.svg
:target: https://pypi.python.org/pypi/pipreqs :target: https://pypi.python.org/pypi/pipreqs
@ -32,7 +31,11 @@ Usage
:: ::
Usage: Usage:
pipreqs [options] <path> pipreqs [options] [<path>]
Arguments:
<path> The path to the directory containing the application files for which a requirements file
should be generated (defaults to the current working directory)
Options: Options:
--use-local Use ONLY local package info instead of querying PyPI --use-local Use ONLY local package info instead of querying PyPI
@ -42,14 +45,18 @@ Usage
$ export HTTP_PROXY="http://10.10.1.10:3128" $ export HTTP_PROXY="http://10.10.1.10:3128"
$ export HTTPS_PROXY="https://10.10.1.10:1080" $ export HTTPS_PROXY="https://10.10.1.10:1080"
--debug Print debug information --debug Print debug information
--ignore <dirs>... Ignore extra directories --ignore <dirs>... Ignore extra directories, each separated by a comma
--no-follow-links Do not follow symbolic links in the project
--encoding <charset> Use encoding parameter for file open --encoding <charset> Use encoding parameter for file open
--savepath <file> Save the list of requirements in the given file --savepath <file> Save the list of requirements in the given file
--print Output the list of requirements in the standard output --print Output the list of requirements in the standard output
--force Overwrite existing requirements.txt --force Overwrite existing requirements.txt
--diff <file> Compare modules in requirements.txt to project imports. --diff <file> Compare modules in requirements.txt to project imports
--clean <file> Clean up requirements.txt by removing modules that are not imported in project. --clean <file> Clean up requirements.txt by removing modules that are not imported in project
--no-pin Omit version of output packages. --mode <scheme> Enables dynamic versioning with <compat>, <gt> or <non-pin> schemes
<compat> | e.g. Flask~=1.1.2
<gt> | e.g. Flask>=1.1.2
<no-pin> | e.g. Flask
Example Example
------- -------
@ -70,5 +77,5 @@ Why not pip freeze?
------------------- -------------------
- ``pip freeze`` only saves the packages that are installed with ``pip install`` in your environment. - ``pip freeze`` only saves the packages that are installed with ``pip install`` in your environment.
- ``pip freeze`` saves all packages in the environment including those that you don't use in your current project. (if you don't have virtualenv) - ``pip freeze`` saves all packages in the environment including those that you don't use in your current project (if you don't have ``virtualenv``).
- and sometimes you just need to create requirements.txt for a new project without installing modules. - and sometimes you just need to create ``requirements.txt`` for a new project without installing modules.

View File

@ -1,3 +1,4 @@
AFQ:pyAFQ
AG_fft_tools:agpy AG_fft_tools:agpy
ANSI:pexpect ANSI:pexpect
Adafruit:Adafruit_Libraries Adafruit:Adafruit_Libraries
@ -13,6 +14,7 @@ Crypto:pycryptodome
Cryptodome:pycryptodomex Cryptodome:pycryptodomex
FSM:pexpect FSM:pexpect
FiftyOneDegrees:51degrees_mobile_detector_v3_wrapper FiftyOneDegrees:51degrees_mobile_detector_v3_wrapper
functional:pyfunctional
GeoBaseMain:GeoBasesDev GeoBaseMain:GeoBasesDev
GeoBases:GeoBasesDev GeoBases:GeoBasesDev
Globals:Zope2 Globals:Zope2
@ -22,6 +24,7 @@ Kittens:astro_kittens
Levenshtein:python_Levenshtein Levenshtein:python_Levenshtein
Lifetime:Zope2 Lifetime:Zope2
MethodObject:ExtensionClass MethodObject:ExtensionClass
MySQLdb:MySQL-python
OFS:Zope2 OFS:Zope2
OpenGL:PyOpenGL OpenGL:PyOpenGL
OpenSSL:pyOpenSSL OpenSSL:pyOpenSSL
@ -592,6 +595,7 @@ devtools:tg.devtools
dgis:2gis dgis:2gis
dhtmlparser:pyDHTMLParser dhtmlparser:pyDHTMLParser
digitalocean:python_digitalocean digitalocean:python_digitalocean
discord:discord.py
distribute_setup:ez_setup distribute_setup:ez_setup
distutils2:Distutils2 distutils2:Distutils2
django:Django django:Django
@ -675,6 +679,7 @@ geventwebsocket:gevent_websocket
gflags:python_gflags gflags:python_gflags
git:GitPython git:GitPython
github:PyGithub github:PyGithub
github3:github3.py
gitpy:git_py gitpy:git_py
globusonline:globusonline_transfer_api_client globusonline:globusonline_transfer_api_client
google:protobuf google:protobuf
@ -701,7 +706,7 @@ html:pies2overrides
htmloutput:nosehtmloutput htmloutput:nosehtmloutput
http:pies2overrides http:pies2overrides
hvad:django_hvad hvad:django_hvad
krbV:krbv hydra:hydra-core
i99fix:199Fix i99fix:199Fix
igraph:python_igraph igraph:python_igraph
imdb:IMDbPY imdb:IMDbPY
@ -727,6 +732,7 @@ keyczar:python_keyczar
keyedcache:django_keyedcache keyedcache:django_keyedcache
keystoneclient:python_keystoneclient keystoneclient:python_keystoneclient
kickstarter:kickstart kickstarter:kickstart
krbv:krbV
kss:kss.core kss:kss.core
kuyruk:Kuyruk kuyruk:Kuyruk
langconv:AdvancedLangConv langconv:AdvancedLangConv
@ -798,7 +804,6 @@ msgpack:msgpack_python
mutations:aino_mutations mutations:aino_mutations
mws:amazon_mws mws:amazon_mws
mysql:mysql_connector_repackaged mysql:mysql_connector_repackaged
MySQL-python:MySQLdb
native_tags:django_native_tags native_tags:django_native_tags
ndg:ndg_httpsclient ndg:ndg_httpsclient
nereid:trytond_nereid nereid:trytond_nereid
@ -999,6 +1004,7 @@ ruamel:ruamel.base
s2repoze:pysaml2 s2repoze:pysaml2
saga:saga_python saga:saga_python
saml2:pysaml2 saml2:pysaml2
samtranslator:aws-sam-translator
sass:libsass sass:libsass
sassc:libsass sassc:libsass
sasstests:libsass sasstests:libsass

84
pipreqs/pipreqs.py Executable file → Normal file
View File

@ -1,5 +1,4 @@
#!/usr/bin/env python #!/usr/bin/env python
# -*- coding: utf-8 -*-
"""pipreqs - Generate pip requirements.txt file based on imports """pipreqs - Generate pip requirements.txt file based on imports
Usage: Usage:
@ -19,27 +18,29 @@ Options:
parameter in your terminal: parameter in your terminal:
$ export HTTP_PROXY="http://10.10.1.10:3128" $ export HTTP_PROXY="http://10.10.1.10:3128"
$ export HTTPS_PROXY="https://10.10.1.10:1080" $ export HTTPS_PROXY="https://10.10.1.10:1080"
--debug Print debug information. --debug Print debug information
--ignore <dirs>... Ignore extra directories, each separated by a comma. --ignore <dirs>... Ignore extra directories, each separated by a comma
--no-follow-links Do not follow symbolic links in the project --no-follow-links Do not follow symbolic links in the project
--encoding <charset> Use encoding parameter for file open --encoding <charset> Use encoding parameter for file open
--savepath <file> Save the list of requirements in the given file --savepath <file> Save the list of requirements in the given file
--print Output the list of requirements in the standard --print Output the list of requirements in the standard
output. output
--force Overwrite existing requirements.txt --force Overwrite existing requirements.txt
--diff <file> Compare modules in requirements.txt to project --diff <file> Compare modules in requirements.txt to project
imports. imports
--clean <file> Clean up requirements.txt by removing modules --clean <file> Clean up requirements.txt by removing modules
that are not imported in project. that are not imported in project
--no-pin Omit version of output packages. --mode <scheme> Enables dynamic versioning with <compat>,
<gt> or <non-pin> schemes.
<compat> | e.g. Flask~=1.1.2
<gt> | e.g. Flask>=1.1.2
<no-pin> | e.g. Flask
""" """
from __future__ import print_function, absolute_import
from contextlib import contextmanager from contextlib import contextmanager
import os import os
import sys import sys
import re import re
import logging import logging
import codecs
import ast import ast
import traceback import traceback
from docopt import docopt from docopt import docopt
@ -54,14 +55,6 @@ REGEXP = [
re.compile(r'^from ((?!\.+).*?) import (?:.*)$') re.compile(r'^from ((?!\.+).*?) import (?:.*)$')
] ]
if sys.version_info[0] > 2:
open_func = open
py2 = False
else:
open_func = codecs.open
py2 = True
py2_exclude = ["concurrent", "concurrent.futures"]
@contextmanager @contextmanager
def _open(filename=None, mode='r'): def _open(filename=None, mode='r'):
@ -118,7 +111,7 @@ def get_all_imports(
candidates += [os.path.splitext(fn)[0] for fn in files] candidates += [os.path.splitext(fn)[0] for fn in files]
for file_name in files: for file_name in files:
file_name = os.path.join(root, file_name) file_name = os.path.join(root, file_name)
with open_func(file_name, "r", encoding=encoding) as f: with open(file_name, "r", encoding=encoding) as f:
contents = f.read() contents = f.read()
try: try:
tree = ast.parse(contents) tree = ast.parse(contents)
@ -153,29 +146,28 @@ def get_all_imports(
with open(join("stdlib"), "r") as f: with open(join("stdlib"), "r") as f:
data = {x.strip() for x in f} data = {x.strip() for x in f}
data = {x for x in data if x not in py2_exclude} if py2 else data
return list(packages - data) return list(packages - data)
def filter_line(l): def filter_line(line):
return len(l) > 0 and l[0] != "#" return len(line) > 0 and line[0] != "#"
def generate_requirements_file(path, imports): def generate_requirements_file(path, imports, symbol):
with _open(path, "w") as out_file: with _open(path, "w") as out_file:
logging.debug('Writing {num} requirements: {imports} to {file}'.format( logging.debug('Writing {num} requirements: {imports} to {file}'.format(
num=len(imports), num=len(imports),
file=path, file=path,
imports=", ".join([x['name'] for x in imports]) imports=", ".join([x['name'] for x in imports])
)) ))
fmt = '{name}=={version}' fmt = '{name}' + symbol + '{version}'
out_file.write('\n'.join( out_file.write('\n'.join(
fmt.format(**item) if item['version'] else '{name}'.format(**item) fmt.format(**item) if item['version'] else '{name}'.format(**item)
for item in imports) + '\n') for item in imports) + '\n')
def output_requirements(imports): def output_requirements(imports, symbol):
generate_requirements_file('-', imports) generate_requirements_file('-', imports, symbol)
def get_imports_info( def get_imports_info(
@ -210,7 +202,7 @@ def get_locally_installed_packages(encoding=None):
for item in files: for item in files:
if "top_level" in item: if "top_level" in item:
item = os.path.join(root, item) item = os.path.join(root, item)
with open_func(item, "r", encoding=encoding) as f: with open(item, "r", encoding=encoding) as f:
package = root.split(os.sep)[-1].split("-") package = root.split(os.sep)[-1].split("-")
try: try:
package_import = f.read().strip().split("\n") package_import = f.read().strip().split("\n")
@ -305,7 +297,7 @@ def parse_requirements(file_):
delim = ["<", ">", "=", "!", "~"] delim = ["<", ">", "=", "!", "~"]
try: try:
f = open_func(file_, "r") f = open(file_, "r")
except OSError: except OSError:
logging.error("Failed on file: {}".format(file_)) logging.error("Failed on file: {}".format(file_))
raise raise
@ -368,11 +360,16 @@ def diff(file_, imports):
def clean(file_, imports): def clean(file_, imports):
"""Remove modules that aren't imported in project from file.""" """Remove modules that aren't imported in project from file."""
modules_not_imported = compare_modules(file_, imports) modules_not_imported = compare_modules(file_, imports)
if len(modules_not_imported) == 0:
logging.info("Nothing to clean in " + file_)
return
re_remove = re.compile("|".join(modules_not_imported)) re_remove = re.compile("|".join(modules_not_imported))
to_write = [] to_write = []
try: try:
f = open_func(file_, "r+") f = open(file_, "r+")
except OSError: except OSError:
logging.error("Failed on file: {}".format(file_)) logging.error("Failed on file: {}".format(file_))
raise raise
@ -392,6 +389,18 @@ def clean(file_, imports):
logging.info("Successfully cleaned up requirements in " + file_) logging.info("Successfully cleaned up requirements in " + file_)
def dynamic_versioning(scheme, imports):
"""Enables dynamic versioning with <compat>, <gt> or <non-pin> schemes."""
if scheme == "no-pin":
imports = [{"name": item["name"], "version": ""} for item in imports]
symbol = ""
elif scheme == "gt":
symbol = ">="
elif scheme == "compat":
symbol = "~="
return imports, symbol
def init(args): def init(args):
encoding = args.get('--encoding') encoding = args.get('--encoding')
extra_ignore_dirs = args.get('--ignore') extra_ignore_dirs = args.get('--ignore')
@ -430,6 +439,8 @@ def init(args):
imports = local + get_imports_info(difference, imports = local + get_imports_info(difference,
proxy=proxy, proxy=proxy,
pypi_server=pypi_server) pypi_server=pypi_server)
# sort imports based on lowercase name of package, similar to `pip freeze`.
imports = sorted(imports, key=lambda x: x['name'].lower())
path = (args["--savepath"] if args["--savepath"] else path = (args["--savepath"] if args["--savepath"] else
os.path.join(input_path, "requirements.txt")) os.path.join(input_path, "requirements.txt"))
@ -446,18 +457,25 @@ def init(args):
and not args["--savepath"] and not args["--savepath"]
and not args["--force"] and not args["--force"]
and os.path.exists(path)): and os.path.exists(path)):
logging.warning("Requirements.txt already exists, " logging.warning("requirements.txt already exists, "
"use --force to overwrite it") "use --force to overwrite it")
return return
if args.get('--no-pin'): if args["--mode"]:
imports = [{'name': item["name"], 'version': ''} for item in imports] scheme = args.get("--mode")
if scheme in ["compat", "gt", "no-pin"]:
imports, symbol = dynamic_versioning(scheme, imports)
else:
raise ValueError("Invalid argument for mode flag, "
"use 'compat', 'gt' or 'no-pin' instead")
else:
symbol = "=="
if args["--print"]: if args["--print"]:
output_requirements(imports) output_requirements(imports, symbol)
logging.info("Successfully output requirements") logging.info("Successfully output requirements")
else: else:
generate_requirements_file(path, imports) generate_requirements_file(path, imports, symbol)
logging.info("Successfully saved requirements file in " + path) logging.info("Successfully saved requirements file in " + path)

File diff suppressed because it is too large Load Diff

View File

@ -1,6 +1,4 @@
#!/usr/bin/env python #!/usr/bin/env python
# -*- coding: utf-8 -*-
try: try:
from setuptools import setup from setuptools import setup
@ -34,7 +32,7 @@ setup(
package_dir={'pipreqs': package_dir={'pipreqs':
'pipreqs'}, 'pipreqs'},
include_package_data=True, include_package_data=True,
package_data={'': ['stdlib','mapping']}, package_data={'': ['stdlib', 'mapping']},
install_requires=requirements, install_requires=requirements,
license='Apache License', license='Apache License',
zip_safe=False, zip_safe=False,
@ -44,12 +42,11 @@ setup(
'Intended Audience :: Developers', 'Intended Audience :: Developers',
'License :: OSI Approved :: Apache Software License', 'License :: OSI Approved :: Apache Software License',
'Natural Language :: English', 'Natural Language :: English',
'Programming Language :: Python :: 2',
'Programming Language :: Python :: 2.7',
'Programming Language :: Python :: 3', 'Programming Language :: Python :: 3',
'Programming Language :: Python :: 3.4',
'Programming Language :: Python :: 3.5',
'Programming Language :: Python :: 3.6', 'Programming Language :: Python :: 3.6',
'Programming Language :: Python :: 3.7',
'Programming Language :: Python :: 3.8',
'Programming Language :: Python :: 3.9',
], ],
test_suite='tests', test_suite='tests',
entry_points={ entry_points={

View File

@ -31,6 +31,10 @@ from pyflakes.test.test_imports import Test as TestImports
# Nose # Nose
from nose.importer import Importer, add_path, remove_path # loader.py from nose.importer import Importer, add_path, remove_path # loader.py
# see issue #88
import analytics
import flask_seasurf
import atexit import atexit
from __future__ import print_function from __future__ import print_function
from docopt import docopt from docopt import docopt

65
tests/_data_clean/test.py Normal file
View File

@ -0,0 +1,65 @@
"""unused import"""
# pylint: disable=undefined-all-variable, import-error, no-absolute-import, too-few-public-methods, missing-docstring
import xml.etree # [unused-import]
import xml.sax # [unused-import]
import os.path as test # [unused-import]
from sys import argv as test2 # [unused-import]
from sys import flags # [unused-import]
# +1:[unused-import,unused-import]
from collections import deque, OrderedDict, Counter
# All imports above should be ignored
import requests # [unused-import]
# setuptools
import zipimport # command/easy_install.py
# twisted
from importlib import invalidate_caches # python/test/test_deprecate.py
# astroid
import zipimport # manager.py
# IPython
from importlib.machinery import all_suffixes # core/completerlib.py
import importlib # html/notebookapp.py
from IPython.utils.importstring import import_item # Many files
# pyflakes
# test/test_doctests.py
from pyflakes.test.test_imports import Test as TestImports
# Nose
from nose.importer import Importer, add_path, remove_path # loader.py
# see issue #88
import analytics
import flask_seasurf
import atexit
from __future__ import print_function
from docopt import docopt
import curses, logging, sqlite3
import logging
import os
import sqlite3
import time
import sys
import signal
import bs4
import nonexistendmodule
import boto as b, peewee as p
# import django
import flask.ext.somext # # #
# from sqlalchemy import model
try:
import ujson as json
except ImportError:
import json
import models
def main():
pass
import after_method_is_valid_even_if_not_pep8

202
tests/test_pipreqs.py Executable file → Normal file
View File

@ -18,22 +18,40 @@ from pipreqs import pipreqs
class TestPipreqs(unittest.TestCase): class TestPipreqs(unittest.TestCase):
def setUp(self): def setUp(self):
self.modules = ['flask', 'requests', 'sqlalchemy', self.modules = [
'docopt', 'boto', 'ipython', 'pyflakes', 'nose', 'flask', 'requests', 'sqlalchemy', 'docopt', 'boto', 'ipython',
'peewee', 'ujson', 'nonexistendmodule', 'bs4', 'after_method_is_valid_even_if_not_pep8' ] 'pyflakes', 'nose', 'analytics', 'flask_seasurf', 'peewee',
'ujson', 'nonexistendmodule', 'bs4',
'after_method_is_valid_even_if_not_pep8'
]
self.modules2 = ['beautifulsoup4'] self.modules2 = ['beautifulsoup4']
self.local = ["docopt", "requests", "nose", 'pyflakes'] self.local = ["docopt", "requests", "nose", 'pyflakes']
self.project = os.path.join(os.path.dirname(__file__), "_data") self.project = os.path.join(os.path.dirname(__file__), "_data")
self.project_invalid = os.path.join(os.path.dirname(__file__), "_invalid_data") self.project_clean = os.path.join(
self.project_with_ignore_directory = os.path.join(os.path.dirname(__file__), "_data_ignore") os.path.dirname(__file__),
self.project_with_duplicated_deps = os.path.join(os.path.dirname(__file__), "_data_duplicated_deps") "_data_clean"
)
self.project_invalid = os.path.join(
os.path.dirname(__file__),
"_invalid_data"
)
self.project_with_ignore_directory = os.path.join(
os.path.dirname(__file__),
"_data_ignore"
)
self.project_with_duplicated_deps = os.path.join(
os.path.dirname(__file__),
"_data_duplicated_deps"
)
self.requirements_path = os.path.join(self.project, "requirements.txt") self.requirements_path = os.path.join(self.project, "requirements.txt")
self.alt_requirement_path = os.path.join( self.alt_requirement_path = os.path.join(
self.project, "requirements2.txt") self.project,
"requirements2.txt"
)
def test_get_all_imports(self): def test_get_all_imports(self):
imports = pipreqs.get_all_imports(self.project) imports = pipreqs.get_all_imports(self.project)
self.assertEqual(len(imports), 13) self.assertEqual(len(imports), 15)
for item in imports: for item in imports:
self.assertTrue( self.assertTrue(
item.lower() in self.modules, "Import is missing: " + item) item.lower() in self.modules, "Import is missing: " + item)
@ -54,7 +72,8 @@ class TestPipreqs(unittest.TestCase):
""" """
Test that invalid python files cannot be imported. Test that invalid python files cannot be imported.
""" """
self.assertRaises(SyntaxError, pipreqs.get_all_imports, self.project_invalid) self.assertRaises(
SyntaxError, pipreqs.get_all_imports, self.project_invalid)
def test_get_imports_info(self): def test_get_imports_info(self):
""" """
@ -62,8 +81,9 @@ class TestPipreqs(unittest.TestCase):
""" """
imports = pipreqs.get_all_imports(self.project) imports = pipreqs.get_all_imports(self.project)
with_info = pipreqs.get_imports_info(imports) with_info = pipreqs.get_imports_info(imports)
# Should contain 10 items without the "nonexistendmodule" and "after_method_is_valid_even_if_not_pep8" # Should contain 10 items without the "nonexistendmodule" and
self.assertEqual(len(with_info), 11) # "after_method_is_valid_even_if_not_pep8"
self.assertEqual(len(with_info), 13)
for item in with_info: for item in with_info:
self.assertTrue( self.assertTrue(
item['name'].lower() in self.modules, item['name'].lower() in self.modules,
@ -77,10 +97,12 @@ class TestPipreqs(unittest.TestCase):
def test_get_use_local_only(self): def test_get_use_local_only(self):
""" """
Test without checking PyPI, check to see if names of local imports matches what we expect Test without checking PyPI, check to see if names of local
imports matches what we expect
- Note even though pyflakes isn't in requirements.txt, - Note even though pyflakes isn't in requirements.txt,
It's added to locals since it is a development dependency for testing It's added to locals since it is a development dependency
for testing
""" """
# should find only docopt and requests # should find only docopt and requests
imports_with_info = pipreqs.get_import_local(self.modules) imports_with_info = pipreqs.get_import_local(self.modules)
@ -89,24 +111,28 @@ class TestPipreqs(unittest.TestCase):
def test_init(self): def test_init(self):
""" """
Test that all modules we will test upon, are in requirements file Test that all modules we will test upon are in requirements file
""" """
pipreqs.init({'<path>': self.project, '--savepath': None, '--print': False, pipreqs.init({'<path>': self.project, '--savepath': None, '--print': False,
'--use-local': None, '--force': True, '--proxy':None, '--pypi-server':None, '--use-local': None, '--force': True, '--proxy':None, '--pypi-server':None,
'--diff': None, '--clean': None}) '--diff': None, '--clean': None, '--mode': None})
assert os.path.exists(self.requirements_path) == 1 assert os.path.exists(self.requirements_path) == 1
with open(self.requirements_path, "r") as f: with open(self.requirements_path, "r") as f:
data = f.read().lower() data = f.read().lower()
for item in self.modules[:-3]: for item in self.modules[:-3]:
self.assertTrue(item.lower() in data) self.assertTrue(item.lower() in data)
# It should be sorted based on names.
data = data.strip().split('\n')
self.assertEqual(data, sorted(data))
def test_init_local_only(self): def test_init_local_only(self):
""" """
Test that items listed in requirements.text are the same as locals expected Test that items listed in requirements.text are the same
as locals expected
""" """
pipreqs.init({'<path>': self.project, '--savepath': None, '--print': False, pipreqs.init({'<path>': self.project, '--savepath': None, '--print': False,
'--use-local': True, '--force': True, '--proxy':None, '--pypi-server':None, '--use-local': True, '--force': True, '--proxy':None, '--pypi-server':None,
'--diff': None, '--clean': None}) '--diff': None, '--clean': None, '--mode': None})
assert os.path.exists(self.requirements_path) == 1 assert os.path.exists(self.requirements_path) == 1
with open(self.requirements_path, "r") as f: with open(self.requirements_path, "r") as f:
data = f.readlines() data = f.readlines()
@ -116,11 +142,12 @@ class TestPipreqs(unittest.TestCase):
def test_init_savepath(self): def test_init_savepath(self):
""" """
Test that we can save requiremnts.tt correctly to a different path Test that we can save requirements.txt correctly
to a different path
""" """
pipreqs.init({'<path>': self.project, '--savepath': pipreqs.init({'<path>': self.project, '--savepath': self.alt_requirement_path,
self.alt_requirement_path, '--use-local': None, '--proxy':None, '--pypi-server':None, '--print': False, '--use-local': None, '--proxy':None, '--pypi-server':None, '--print': False,
"--diff": None, "--clean": None}) '--diff': None, '--clean': None, '--mode': None})
assert os.path.exists(self.alt_requirement_path) == 1 assert os.path.exists(self.alt_requirement_path) == 1
with open(self.alt_requirement_path, "r") as f: with open(self.alt_requirement_path, "r") as f:
data = f.read().lower() data = f.read().lower()
@ -131,13 +158,14 @@ class TestPipreqs(unittest.TestCase):
def test_init_overwrite(self): def test_init_overwrite(self):
""" """
Test that if requiremnts.txt exists, it will not automatically be overwritten Test that if requiremnts.txt exists, it will not be
automatically overwritten
""" """
with open(self.requirements_path, "w") as f: with open(self.requirements_path, "w") as f:
f.write("should_not_be_overwritten") f.write("should_not_be_overwritten")
pipreqs.init({'<path>': self.project, '--savepath': None, pipreqs.init({'<path>': self.project, '--savepath': None, '--use-local': None,
'--use-local': None, '--force': None, '--proxy':None, '--pypi-server':None, '--print': False, '--force': None, '--proxy':None, '--pypi-server':None, '--print': False,
"--diff": None, "--clean": None}) '--diff': None, '--clean': None, '--mode': None})
assert os.path.exists(self.requirements_path) == 1 assert os.path.exists(self.requirements_path) == 1
with open(self.requirements_path, "r") as f: with open(self.requirements_path, "r") as f:
data = f.read().lower() data = f.read().lower()
@ -145,35 +173,43 @@ class TestPipreqs(unittest.TestCase):
def test_get_import_name_without_alias(self): def test_get_import_name_without_alias(self):
""" """
Test that function get_name_without_alias() will work on a string. Test that function get_name_without_alias()
- Note: This isn't truly needed when pipreqs is walking the AST to find imports will work on a string.
- Note: This isn't truly needed when pipreqs is walking
the AST to find imports
""" """
import_name_with_alias = "requests as R" import_name_with_alias = "requests as R"
expected_import_name_without_alias = "requests" expected_import_name_without_alias = "requests"
import_name_without_aliases = pipreqs.get_name_without_alias( import_name_without_aliases = pipreqs.get_name_without_alias(
import_name_with_alias) import_name_with_alias)
self.assertEqual( self.assertEqual(
import_name_without_aliases, expected_import_name_without_alias) import_name_without_aliases,
expected_import_name_without_alias
)
def test_custom_pypi_server(self): def test_custom_pypi_server(self):
""" """
Test that trying to get a custom pypi sever fails correctly Test that trying to get a custom pypi sever fails correctly
""" """
self.assertRaises(requests.exceptions.MissingSchema, pipreqs.init, {'<path>': self.project, '--savepath': None, '--print': False, self.assertRaises(
'--use-local': None, '--force': True, '--proxy': None, '--pypi-server': 'nonexistent'}) requests.exceptions.MissingSchema, pipreqs.init,
{'<path>': self.project, '--savepath': None, '--print': False,
'--use-local': None, '--force': True, '--proxy': None,
'--pypi-server': 'nonexistent'}
)
def test_ignored_directory(self): def test_ignored_directory(self):
""" """
Test --ignore parameter Test --ignore parameter
""" """
pipreqs.init( pipreqs.init(
{'<path>': self.project_with_ignore_directory, '--savepath': None, '--print': False, {'<path>': self.project_with_ignore_directory, '--savepath': None,
'--use-local': None, '--force': True, '--print': False, '--use-local': None, '--force': True,
'--proxy':None, '--proxy':None, '--pypi-server':None,
'--pypi-server':None,
'--ignore':'.ignored_dir,.ignore_second', '--ignore':'.ignored_dir,.ignore_second',
'--diff': None, '--diff': None,
'--clean': None '--clean': None,
'--mode': None
} }
) )
with open(os.path.join(self.project_with_ignore_directory, "requirements.txt"), "r") as f: with open(os.path.join(self.project_with_ignore_directory, "requirements.txt"), "r") as f:
@ -181,9 +217,27 @@ class TestPipreqs(unittest.TestCase):
for item in ['click', 'getpass']: for item in ['click', 'getpass']:
self.assertFalse(item.lower() in data) self.assertFalse(item.lower() in data)
def test_omit_version(self): def test_dynamic_version_no_pin_scheme(self):
""" """
Test --no-pin parameter Test --mode=no-pin
"""
pipreqs.init(
{'<path>': self.project_with_ignore_directory, '--savepath': None,
'--print': False, '--use-local': None, '--force': True,
'--proxy': None, '--pypi-server': None,
'--diff': None,
'--clean': None,
'--mode': 'no-pin'
}
)
with open(os.path.join(self.project_with_ignore_directory, "requirements.txt"), "r") as f:
data = f.read().lower()
for item in ['beautifulsoup4', 'boto']:
self.assertTrue(item.lower() in data)
def test_dynamic_version_gt_scheme(self):
"""
Test --mode=gt
""" """
pipreqs.init( pipreqs.init(
{'<path>': self.project_with_ignore_directory, '--savepath': None, '--print': False, {'<path>': self.project_with_ignore_directory, '--savepath': None, '--print': False,
@ -192,13 +246,81 @@ class TestPipreqs(unittest.TestCase):
'--pypi-server': None, '--pypi-server': None,
'--diff': None, '--diff': None,
'--clean': None, '--clean': None,
'--no-pin': True '--mode': 'gt'
} }
) )
with open(os.path.join(self.project_with_ignore_directory, "requirements.txt"), "r") as f: with open(os.path.join(self.project_with_ignore_directory, "requirements.txt"), "r") as f:
data = f.readlines()
for item in data:
symbol = '>='
message = 'symbol is not in item'
self.assertIn(symbol, item, message)
def test_dynamic_version_compat_scheme(self):
"""
Test --mode=compat
"""
pipreqs.init(
{'<path>': self.project_with_ignore_directory, '--savepath': None, '--print': False,
'--use-local': None, '--force': True,
'--proxy': None,
'--pypi-server': None,
'--diff': None,
'--clean': None,
'--mode': 'compat'
}
)
with open(os.path.join(self.project_with_ignore_directory, "requirements.txt"), "r") as f:
data = f.readlines()
for item in data:
symbol = '~='
message = 'symbol is not in item'
self.assertIn(symbol, item, message)
def test_clean(self):
"""
Test --clean parameter
"""
pipreqs.init(
{'<path>': self.project, '--savepath': None, '--print': False,
'--use-local': None, '--force': True, '--proxy': None,
'--pypi-server': None, '--diff': None, '--clean': None,
'--mode': None}
)
assert os.path.exists(self.requirements_path) == 1
pipreqs.init(
{'<path>': self.project, '--savepath': None, '--print': False,
'--use-local': None, '--force': None, '--proxy': None,
'--pypi-server': None, '--diff': None,
'--clean': self.requirements_path, '--mode': 'non-pin'}
)
with open(self.requirements_path, "r") as f:
data = f.read().lower() data = f.read().lower()
for item in ['beautifulsoup4==4.8.1', 'boto==2.49.0']: for item in self.modules[:-3]:
self.assertFalse(item.lower() in data) self.assertTrue(item.lower() in data)
def test_clean_with_imports_to_clean(self):
"""
Test --clean parameter when there are imports to clean
"""
cleaned_module = 'sqlalchemy'
pipreqs.init(
{'<path>': self.project, '--savepath': None, '--print': False,
'--use-local': None, '--force': True, '--proxy': None,
'--pypi-server': None, '--diff': None, '--clean': None,
'--mode': None}
)
assert os.path.exists(self.requirements_path) == 1
modules_clean = [m for m in self.modules if m != cleaned_module]
pipreqs.init(
{'<path>': self.project_clean, '--savepath': None,
'--print': False, '--use-local': None, '--force': None,
'--proxy': None, '--pypi-server': None, '--diff': None,
'--clean': self.requirements_path, '--mode': 'non-pin'}
)
with open(self.requirements_path, "r") as f:
data = f.read().lower()
self.assertTrue(cleaned_module not in data)
def tearDown(self): def tearDown(self):
""" """

17
tox.ini
View File

@ -1,5 +1,13 @@
[tox] [tox]
envlist = py27, py34, py35, py36, pypy, flake8 envlist = py36, py37, py38, py39, pypy3, flake8
[gh-actions]
python =
3.6: py36
3.7: py37
3.8: py38
3.9: py39
pypy-3.7: pypy3
[testenv] [testenv]
setenv = setenv =
@ -7,10 +15,3 @@ setenv =
commands = python setup.py test commands = python setup.py test
deps = deps =
-r{toxinidir}/requirements.txt -r{toxinidir}/requirements.txt
[testenv:flake8]
basepython = python3.6
commands = flake8 pipreqs
deps =
-r{toxinidir}/requirements.txt
flake8