github/pipreqs
1
1
Fork 0
mirror of https://github.com/bndr/pipreqs.git synced 2025-06-06 03:25:21 +00:00
Code Issues 3 Packages Projects Releases Wiki Activity

Merge branch 'master' into add_notebook_support

Browse Source
This commit is contained in:
Kazuma (Pakio) Arimura 2022-07-17 10:09:34 +09:00
commit ef4cb12249
16 changed files with 1911 additions and 335 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
.coveralls.yml
View File

@ -1 +0,0 @@
service_name: "travis-ci"

20
.github/workflows/flake8.yml vendored Normal file
View File

@ -0,0 +1,20 @@
name: flake8
on: pull_request
jobs:
flake8-lint:
runs-on: ubuntu-latest
name: Lint
steps:
- name: Check out source repository
uses: actions/checkout@v2
- name: Set up Python environment
uses: actions/setup-python@v2
with:
python-version: "3.9"
- name: flake8 Lint
uses: reviewdog/action-flake8@v3
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
reporter: github-pr-review

50
.github/workflows/tests.yml vendored Normal file
View File

@ -0,0 +1,50 @@
name: Tests and Codecov
on: pull_request
jobs:
run_tests:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
python-version: [3.6, 3.7, 3.8, 3.9, pypy-3.7]
steps:
- name: Checkout repository
uses: actions/checkout@v2
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install tox tox-gh-actions
- name: Test with tox
run: tox
coverage_report:
needs: run_tests
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v2
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install coverage docopt yarg requests
- name: Calculate coverage
run: coverage run --source=pipreqs -m unittest discover
- name: Create XML report
run: coverage xml
- name: Upload coverage to Codecov
uses: codecov/codecov-action@v2
with:
files: coverage.xml
fail_ci_if_error: true

30
.travis.yml
View File

@ -1,30 +0,0 @@
# Config file for automatic testing at travis-ci.org
language: python
matrix:
include:
- python: 3.6
env: TOX_ENV=py36
- python: 3.5
env: TOX_ENV=py35
- python: 3.4
env: TOX_ENV=py34
- python: 2.7
env: TOX_ENV=py27
- python: pypy
env: TOX_ENV=pypy
- python: 3.6
env: TOX_ENV=flake8
# Use tox to run tests on Travis-CI to keep one unified method of running tests in any environment
install:
- pip install coverage coveralls tox
# Command to run tests, e.g. python setup.py test
script: tox -e $TOX_ENV
# Use after_success to get a single coveralls report
after_success:
- coverage run --source=pipreqs setup.py test
- coveralls

3
AUTHORS.rst
View File

@ -10,4 +10,5 @@ Development Lead
Contributors
------------
None yet. Why not be the first?
* Jake Teo <mapattacker@gmail.com>
* Jerome Chan <cjerome94@gmail.com>

5
HISTORY.rst
View File

@ -3,6 +3,11 @@
History
-------
0.4.11 (2020-03-29)
--------------------
* Implement '--mode' (Jake Teo, Jerome Chan)
0.4.8 (2017-06-30)
--------------------

48
README.rst
View File

@ -1,23 +1,22 @@
===============================
=============================================================================
``pipreqs`` - Generate requirements.txt file for any project based on imports
===============================
=============================================================================
.. image:: https://img.shields.io/travis/bndr/pipreqs.svg
:target: https://travis-ci.org/bndr/pipreqs
.. image:: https://img.shields.io/pypi/v/pipreqs.svg
:target: https://pypi.python.org/pypi/pipreqs
.. image:: https://img.shields.io/coveralls/bndr/pipreqs.svg
:target: https://coveralls.io/r/bndr/pipreqs
.. image:: https://img.shields.io/pypi/l/pipreqs.svg
.. image:: https://codecov.io/gh/bndr/pipreqs/branch/master/graph/badge.svg?token=0rfPfUZEAX
:target: https://codecov.io/gh/bndr/pipreqs
.. image:: https://img.shields.io/pypi/l/pipreqs.svg
:target: https://pypi.python.org/pypi/pipreqs
Installation
------------
@ -32,7 +31,11 @@ Usage
::
Usage:
pipreqs [options] <path>
pipreqs [options] [<path>]
Arguments:
<path> The path to the directory containing the application files for which a requirements file
should be generated (defaults to the current working directory)
Options:
--use-local Use ONLY local package info instead of querying PyPI
@ -42,14 +45,19 @@ Usage
$ export HTTP_PROXY="http://10.10.1.10:3128"
$ export HTTPS_PROXY="https://10.10.1.10:1080"
--debug Print debug information
--ignore <dirs>... Ignore extra directories
--ignore <dirs>... Ignore extra directories, each separated by a comma
--no-follow-links Do not follow symbolic links in the project
--encoding <charset> Use encoding parameter for file open
--savepath <file> Save the list of requirements in the given file
--print Output the list of requirements in the standard output
--force Overwrite existing requirements.txt
--diff <file> Compare modules in requirements.txt to project imports.
--clean <file> Clean up requirements.txt by removing modules that are not imported in project.
--no-pin Omit version of output packages.
--diff <file> Compare modules in requirements.txt to project imports
--clean <file> Clean up requirements.txt by removing modules that are not imported in project
--mode <scheme> Enables dynamic versioning with <compat>, <gt> or <non-pin> schemes
<compat> | e.g. Flask~=1.1.2
<gt> | e.g. Flask>=1.1.2
<no-pin> | e.g. Flask
Example
-------
@ -65,10 +73,10 @@ Contents of requirements.txt
wheel==0.23.0
Yarg==0.1.9
docopt==0.6.2
Why not pip freeze?
-------------------
- ``pip freeze`` only saves the packages that are installed with ``pip install`` in your environment.
- ``pip freeze`` saves all packages in the environment including those that you don't use in your current project. (if you don't have virtualenv)
- and sometimes you just need to create requirements.txt for a new project without installing modules.
- ``pip freeze`` only saves the packages that are installed with ``pip install`` in your environment.
- ``pip freeze`` saves all packages in the environment including those that you don't use in your current project (if you don't have ``virtualenv``).
- and sometimes you just need to create ``requirements.txt`` for a new project without installing modules.

4
pipreqs/__init__.py
View File

@ -1,5 +1,3 @@
# -*- coding: utf-8 -*-
__author__ = 'Vadim Kravcenko'
__email__ = 'vadim.kravcenko@gmail.com'
__version__ = '0.4.8'
__version__ = '0.4.11'

10
pipreqs/mapping
View File

@ -1,3 +1,4 @@
AFQ:pyAFQ
AG_fft_tools:agpy
ANSI:pexpect
Adafruit:Adafruit_Libraries
@ -13,6 +14,7 @@ Crypto:pycryptodome
Cryptodome:pycryptodomex
FSM:pexpect
FiftyOneDegrees:51degrees_mobile_detector_v3_wrapper
functional:pyfunctional
GeoBaseMain:GeoBasesDev
GeoBases:GeoBasesDev
Globals:Zope2
@ -22,6 +24,7 @@ Kittens:astro_kittens
Levenshtein:python_Levenshtein
Lifetime:Zope2
MethodObject:ExtensionClass
MySQLdb:MySQL-python
OFS:Zope2
OpenGL:PyOpenGL
OpenSSL:pyOpenSSL
@ -592,6 +595,7 @@ devtools:tg.devtools
dgis:2gis
dhtmlparser:pyDHTMLParser
digitalocean:python_digitalocean
discord:discord.py
distribute_setup:ez_setup
distutils2:Distutils2
django:Django
@ -675,6 +679,7 @@ geventwebsocket:gevent_websocket
gflags:python_gflags
git:GitPython
github:PyGithub
github3:github3.py
gitpy:git_py
globusonline:globusonline_transfer_api_client
google:protobuf
@ -701,7 +706,7 @@ html:pies2overrides
htmloutput:nosehtmloutput
http:pies2overrides
hvad:django_hvad
krbV:krbv
hydra:hydra-core
i99fix:199Fix
igraph:python_igraph
imdb:IMDbPY
@ -727,6 +732,7 @@ keyczar:python_keyczar
keyedcache:django_keyedcache
keystoneclient:python_keystoneclient
kickstarter:kickstart
krbv:krbV
kss:kss.core
kuyruk:Kuyruk
langconv:AdvancedLangConv
@ -798,7 +804,6 @@ msgpack:msgpack_python
mutations:aino_mutations
mws:amazon_mws
mysql:mysql_connector_repackaged
MySQL-python:MySQLdb
native_tags:django_native_tags
ndg:ndg_httpsclient
nereid:trytond_nereid
@ -999,6 +1004,7 @@ ruamel:ruamel.base
s2repoze:pysaml2
saga:saga_python
saml2:pysaml2
samtranslator:aws-sam-translator
sass:libsass
sassc:libsass
sasstests:libsass

83
pipreqs/pipreqs.py Executable file → Normal file
View File

@ -1,5 +1,4 @@
#!/usr/bin/env python
# -*- coding: utf-8 -*-
"""pipreqs - Generate pip requirements.txt file based on imports
Usage:
@ -19,27 +18,29 @@ Options:
parameter in your terminal:
$ export HTTP_PROXY="http://10.10.1.10:3128"
$ export HTTPS_PROXY="https://10.10.1.10:1080"
--debug Print debug information.
--ignore <dirs>... Ignore extra directories, each separated by a comma.
--debug Print debug information
--ignore <dirs>... Ignore extra directories, each separated by a comma
--no-follow-links Do not follow symbolic links in the project
--encoding <charset> Use encoding parameter for file open
--savepath <file> Save the list of requirements in the given file
--print Output the list of requirements in the standard
output.
output
--force Overwrite existing requirements.txt
--diff <file> Compare modules in requirements.txt to project
imports.
imports
--clean <file> Clean up requirements.txt by removing modules
that are not imported in project.
--no-pin Omit version of output packages.
that are not imported in project
--mode <scheme> Enables dynamic versioning with <compat>,
<gt> or <non-pin> schemes.
<compat> | e.g. Flask~=1.1.2
<gt> | e.g. Flask>=1.1.2
<no-pin> | e.g. Flask
"""
from __future__ import print_function, absolute_import
from contextlib import contextmanager
import os
import sys
import re
import logging
import codecs
import ast
import traceback
from docopt import docopt
@ -55,14 +56,6 @@ REGEXP = [
re.compile(r'^from ((?!\.+).*?) import (?:.*)$')
]
if sys.version_info[0] > 2:
open_func = open
py2 = False
else:
open_func = codecs.open
py2 = True
py2_exclude = ["concurrent", "concurrent.futures"]
@contextmanager
def _open(filename=None, mode='r'):
@ -126,7 +119,7 @@ def get_all_imports(
file_name = os.path.join(root, file_name)
contents = ''
if filter_ext(file_name, [".py"]):
with open_func(file_name, "r", encoding=encoding) as f:
with open(file_name, "r", encoding=encoding) as f:
contents = f.read()
elif filter_ext(file_name, [".ipynb"]):
contents = ipynb_2_py(file_name, encoding=encoding)
@ -167,7 +160,6 @@ def get_all_imports(
with open(join("stdlib"), "r") as f:
data = {x.strip() for x in f}
data = {x for x in data if x not in py2_exclude} if py2 else data
return list(packages - data)
@ -184,7 +176,7 @@ def ipynb_2_py(file_name, encoding=None):
Args:
file_name (str): notebook file path to parse as python script
encoding (str): encoding of file
encoding (str): file encoding
Returns:
str: parsed string
@ -196,22 +188,21 @@ def ipynb_2_py(file_name, encoding=None):
return body.encode(encoding if encoding is not None else "utf-8")
def generate_requirements_file(path, imports):
def generate_requirements_file(path, imports, symbol):
with _open(path, "w") as out_file:
logging.debug('Writing {num} requirements: {imports} to {file}'.format(
num=len(imports),
file=path,
imports=", ".join([x['name'] for x in imports])
))
fmt = '{name}=={version}'
fmt = '{name}' + symbol + '{version}'
out_file.write('\n'.join(
fmt.format(**item) if item['version'] else '{name}'.format(**item)
for item in imports) + '\n')
def output_requirements(imports):
generate_requirements_file('-', imports)
def output_requirements(imports, symbol):
generate_requirements_file('-', imports, symbol)
def get_imports_info(
@ -246,7 +237,7 @@ def get_locally_installed_packages(encoding=None):
for item in files:
if "top_level" in item:
item = os.path.join(root, item)
with open_func(item, "r", encoding=encoding) as f:
with open(item, "r", encoding=encoding) as f:
package = root.split(os.sep)[-1].split("-")
try:
package_import = f.read().strip().split("\n")
@ -341,7 +332,7 @@ def parse_requirements(file_):
delim = ["<", ">", "=", "!", "~"]
try:
f = open_func(file_, "r")
f = open(file_, "r")
except OSError:
logging.error("Failed on file: {}".format(file_))
raise
@ -404,11 +395,16 @@ def diff(file_, imports):
def clean(file_, imports):
"""Remove modules that aren't imported in project from file."""
modules_not_imported = compare_modules(file_, imports)
if len(modules_not_imported) == 0:
logging.info("Nothing to clean in " + file_)
return
re_remove = re.compile("|".join(modules_not_imported))
to_write = []
try:
f = open_func(file_, "r+")
f = open(file_, "r+")
except OSError:
logging.error("Failed on file: {}".format(file_))
raise
@ -428,6 +424,18 @@ def clean(file_, imports):
logging.info("Successfully cleaned up requirements in " + file_)
def dynamic_versioning(scheme, imports):
"""Enables dynamic versioning with <compat>, <gt> or <non-pin> schemes."""
if scheme == "no-pin":
imports = [{"name": item["name"], "version": ""} for item in imports]
symbol = ""
elif scheme == "gt":
symbol = ">="
elif scheme == "compat":
symbol = "~="
return imports, symbol
def init(args):
encoding = args.get('--encoding')
extra_ignore_dirs = args.get('--ignore')
@ -466,6 +474,8 @@ def init(args):
imports = local + get_imports_info(difference,
proxy=proxy,
pypi_server=pypi_server)
# sort imports based on lowercase name of package, similar to `pip freeze`.
imports = sorted(imports, key=lambda x: x['name'].lower())
path = (args["--savepath"] if args["--savepath"] else
os.path.join(input_path, "requirements.txt"))
@ -482,18 +492,25 @@ def init(args):
and not args["--savepath"]
and not args["--force"]
and os.path.exists(path)):
logging.warning("Requirements.txt already exists, "
logging.warning("requirements.txt already exists, "
"use --force to overwrite it")
return
if args.get('--no-pin'):
imports = [{'name': item["name"], 'version': ''} for item in imports]
if args["--mode"]:
scheme = args.get("--mode")
if scheme in ["compat", "gt", "no-pin"]:
imports, symbol = dynamic_versioning(scheme, imports)
else:
raise ValueError("Invalid argument for mode flag, "
"use 'compat', 'gt' or 'no-pin' instead")
else:
symbol = "=="
if args["--print"]:
output_requirements(imports)
output_requirements(imports, symbol)
logging.info("Successfully output requirements")
else:
generate_requirements_file(path, imports)
generate_requirements_file(path, imports, symbol)
logging.info("Successfully saved requirements file in " + path)

1654
pipreqs/stdlib
View File

File diff suppressed because it is too large Load Diff

11
setup.py
View File

@ -1,6 +1,4 @@
#!/usr/bin/env python
# -*- coding: utf-8 -*-
try:
from setuptools import setup
@ -34,7 +32,7 @@ setup(
package_dir={'pipreqs':
'pipreqs'},
include_package_data=True,
package_data={'': ['stdlib','mapping']},
package_data={'': ['stdlib', 'mapping']},
install_requires=requirements,
license='Apache License',
zip_safe=False,
@ -44,12 +42,11 @@ setup(
'Intended Audience :: Developers',
'License :: OSI Approved :: Apache Software License',
'Natural Language :: English',
'Programming Language :: Python :: 2',
'Programming Language :: Python :: 2.7',
'Programming Language :: Python :: 3',
'Programming Language :: Python :: 3.4',
'Programming Language :: Python :: 3.5',
'Programming Language :: Python :: 3.6',
'Programming Language :: Python :: 3.7',
'Programming Language :: Python :: 3.8',
'Programming Language :: Python :: 3.9',
],
test_suite='tests',
entry_points={

4
tests/_data/test.py
View File

@ -31,6 +31,10 @@ from pyflakes.test.test_imports import Test as TestImports
# Nose
from nose.importer import Importer, add_path, remove_path # loader.py
# see issue #88
import analytics
import flask_seasurf
import atexit
from __future__ import print_function
from docopt import docopt

65
tests/_data_clean/test.py Normal file
View File

@ -0,0 +1,65 @@
"""unused import"""
# pylint: disable=undefined-all-variable, import-error, no-absolute-import, too-few-public-methods, missing-docstring
import xml.etree # [unused-import]
import xml.sax # [unused-import]
import os.path as test # [unused-import]
from sys import argv as test2 # [unused-import]
from sys import flags # [unused-import]
# +1:[unused-import,unused-import]
from collections import deque, OrderedDict, Counter
# All imports above should be ignored
import requests # [unused-import]
# setuptools
import zipimport # command/easy_install.py
# twisted
from importlib import invalidate_caches # python/test/test_deprecate.py
# astroid
import zipimport # manager.py
# IPython
from importlib.machinery import all_suffixes # core/completerlib.py
import importlib # html/notebookapp.py
from IPython.utils.importstring import import_item # Many files
# pyflakes
# test/test_doctests.py
from pyflakes.test.test_imports import Test as TestImports
# Nose
from nose.importer import Importer, add_path, remove_path # loader.py
# see issue #88
import analytics
import flask_seasurf
import atexit
from __future__ import print_function
from docopt import docopt
import curses, logging, sqlite3
import logging
import os
import sqlite3
import time
import sys
import signal
import bs4
import nonexistendmodule
import boto as b, peewee as p
# import django
import flask.ext.somext # # #
# from sqlalchemy import model
try:
import ujson as json
except ImportError:
import json
import models
def main():
pass
import after_method_is_valid_even_if_not_pep8

239
tests/test_pipreqs.py Executable file → Normal file
View File

@ -18,26 +18,57 @@ from pipreqs import pipreqs
class TestPipreqs(unittest.TestCase):
def setUp(self):
self.modules = ['flask', 'requests', 'sqlalchemy',
'docopt', 'boto', 'ipython', 'pyflakes', 'nose',
'peewee', 'ujson', 'nonexistendmodule', 'bs4', 'after_method_is_valid_even_if_not_pep8' ]
self.modules = [
'flask', 'requests', 'sqlalchemy', 'docopt', 'boto', 'ipython',
'pyflakes', 'nose', 'analytics', 'flask_seasurf', 'peewee',
'ujson', 'nonexistendmodule', 'bs4',
'after_method_is_valid_even_if_not_pep8'
]
self.modules2 = ['beautifulsoup4']
self.local = ["docopt", "requests", "nose", 'pyflakes']
self.project = os.path.join(os.path.dirname(__file__), "_data")
self.project_invalid = os.path.join(os.path.dirname(__file__), "_invalid_data")
self.project_with_ignore_directory = os.path.join(os.path.dirname(__file__), "_data_ignore")
self.project_with_duplicated_deps = os.path.join(os.path.dirname(__file__), "_data_duplicated_deps")
self.project_with_notebooks = os.path.join(os.path.dirname(__file__), "_data_notebook")
self.project_with_invalid_notebooks = os.path.join(os.path.dirname(__file__), "_invalid_data_notebook")
self.project_clean = os.path.join(
os.path.dirname(__file__),
"_data_clean"
)
self.project_invalid = os.path.join(
os.path.dirname(__file__),
"_invalid_data"
)
self.project_with_ignore_directory = os.path.join(
os.path.dirname(__file__),
"_data_ignore"
)
self.project_with_duplicated_deps = os.path.join(
os.path.dirname(__file__),
"_data_duplicated_deps"
)
self.requirements_path = os.path.join(self.project, "requirements.txt")
self.alt_requirement_path = os.path.join(
self.project, "requirements2.txt")
self.compatible_files_path = {"original": os.path.join(os.path.dirname(__file__), "_data/test.py"),
"notebook": os.path.join(os.path.dirname(__file__), "_data_notebook/test.ipynb")}
self.project,
"requirements2.txt"
)
self.project_with_notebooks = os.path.join(
os.path.dirname(__file__),
"_data_notebook"
)
self.project_with_invalid_notebooks = os.path.join(
os.path.dirname(__file__),
"_invalid_data_notebook"
)
self.compatible_files_path = {
"original": os.path.join(
os.path.dirname(__file__),
"_data/test.py"
),
"notebook": os.path.join(
os.path.dirname(__file__),
"_data_notebook/test.ipynb"
)}
def test_get_all_imports(self):
imports = pipreqs.get_all_imports(self.project)
self.assertEqual(len(imports), 13)
self.assertEqual(len(imports), 15)
for item in imports:
self.assertTrue(
item.lower() in self.modules, "Import is missing: " + item)
@ -58,7 +89,8 @@ class TestPipreqs(unittest.TestCase):
"""
Test that invalid python files cannot be imported.
"""
self.assertRaises(SyntaxError, pipreqs.get_all_imports, self.project_invalid)
self.assertRaises(
SyntaxError, pipreqs.get_all_imports, self.project_invalid)
def test_get_imports_info(self):
"""
@ -66,8 +98,9 @@ class TestPipreqs(unittest.TestCase):
"""
imports = pipreqs.get_all_imports(self.project)
with_info = pipreqs.get_imports_info(imports)
# Should contain 10 items without the "nonexistendmodule" and "after_method_is_valid_even_if_not_pep8"
self.assertEqual(len(with_info), 11)
# Should contain 10 items without the "nonexistendmodule" and
# "after_method_is_valid_even_if_not_pep8"
self.assertEqual(len(with_info), 13)
for item in with_info:
self.assertTrue(
item['name'].lower() in self.modules,
@ -81,10 +114,12 @@ class TestPipreqs(unittest.TestCase):
def test_get_use_local_only(self):
"""
Test without checking PyPI, check to see if names of local imports matches what we expect
Test without checking PyPI, check to see if names of local
imports matches what we expect
- Note even though pyflakes isn't in requirements.txt,
It's added to locals since it is a development dependency for testing
It's added to locals since it is a development dependency
for testing
"""
# should find only docopt and requests
imports_with_info = pipreqs.get_import_local(self.modules)
@ -93,24 +128,28 @@ class TestPipreqs(unittest.TestCase):
def test_init(self):
"""
Test that all modules we will test upon, are in requirements file
Test that all modules we will test upon are in requirements file
"""
pipreqs.init({'<path>': self.project, '--savepath': None, '--print': False,
'--use-local': None, '--force': True, '--proxy':None, '--pypi-server':None,
'--diff': None, '--clean': None})
'--diff': None, '--clean': None, '--mode': None})
assert os.path.exists(self.requirements_path) == 1
with open(self.requirements_path, "r") as f:
data = f.read().lower()
for item in self.modules[:-3]:
self.assertTrue(item.lower() in data)
# It should be sorted based on names.
data = data.strip().split('\n')
self.assertEqual(data, sorted(data))
def test_init_local_only(self):
"""
Test that items listed in requirements.text are the same as locals expected
Test that items listed in requirements.text are the same
as locals expected
"""
pipreqs.init({'<path>': self.project, '--savepath': None, '--print': False,
'--use-local': True, '--force': True, '--proxy':None, '--pypi-server':None,
'--diff': None, '--clean': None})
'--diff': None, '--clean': None, '--mode': None})
assert os.path.exists(self.requirements_path) == 1
with open(self.requirements_path, "r") as f:
data = f.readlines()
@ -120,11 +159,12 @@ class TestPipreqs(unittest.TestCase):
def test_init_savepath(self):
"""
Test that we can save requiremnts.tt correctly to a different path
Test that we can save requirements.txt correctly
to a different path
"""
pipreqs.init({'<path>': self.project, '--savepath':
self.alt_requirement_path, '--use-local': None, '--proxy':None, '--pypi-server':None, '--print': False,
"--diff": None, "--clean": None})
pipreqs.init({'<path>': self.project, '--savepath': self.alt_requirement_path,
'--use-local': None, '--proxy':None, '--pypi-server':None, '--print': False,
'--diff': None, '--clean': None, '--mode': None})
assert os.path.exists(self.alt_requirement_path) == 1
with open(self.alt_requirement_path, "r") as f:
data = f.read().lower()
@ -135,13 +175,14 @@ class TestPipreqs(unittest.TestCase):
def test_init_overwrite(self):
"""
Test that if requiremnts.txt exists, it will not automatically be overwritten
Test that if requiremnts.txt exists, it will not be
automatically overwritten
"""
with open(self.requirements_path, "w") as f:
f.write("should_not_be_overwritten")
pipreqs.init({'<path>': self.project, '--savepath': None,
'--use-local': None, '--force': None, '--proxy':None, '--pypi-server':None, '--print': False,
"--diff": None, "--clean": None})
pipreqs.init({'<path>': self.project, '--savepath': None, '--use-local': None,
'--force': None, '--proxy':None, '--pypi-server':None, '--print': False,
'--diff': None, '--clean': None, '--mode': None})
assert os.path.exists(self.requirements_path) == 1
with open(self.requirements_path, "r") as f:
data = f.read().lower()
@ -149,35 +190,43 @@ class TestPipreqs(unittest.TestCase):
def test_get_import_name_without_alias(self):
"""
Test that function get_name_without_alias() will work on a string.
- Note: This isn't truly needed when pipreqs is walking the AST to find imports
Test that function get_name_without_alias()
will work on a string.
- Note: This isn't truly needed when pipreqs is walking
the AST to find imports
"""
import_name_with_alias = "requests as R"
expected_import_name_without_alias = "requests"
import_name_without_aliases = pipreqs.get_name_without_alias(
import_name_with_alias)
self.assertEqual(
import_name_without_aliases, expected_import_name_without_alias)
import_name_without_aliases,
expected_import_name_without_alias
)
def test_custom_pypi_server(self):
"""
Test that trying to get a custom pypi sever fails correctly
"""
self.assertRaises(requests.exceptions.MissingSchema, pipreqs.init, {'<path>': self.project, '--savepath': None, '--print': False,
'--use-local': None, '--force': True, '--proxy': None, '--pypi-server': 'nonexistent'})
self.assertRaises(
requests.exceptions.MissingSchema, pipreqs.init,
{'<path>': self.project, '--savepath': None, '--print': False,
'--use-local': None, '--force': True, '--proxy': None,
'--pypi-server': 'nonexistent'}
)
def test_ignored_directory(self):
"""
Test --ignore parameter
"""
pipreqs.init(
{'<path>': self.project_with_ignore_directory, '--savepath': None, '--print': False,
'--use-local': None, '--force': True,
'--proxy':None,
'--pypi-server':None,
{'<path>': self.project_with_ignore_directory, '--savepath': None,
'--print': False, '--use-local': None, '--force': True,
'--proxy':None, '--pypi-server':None,
'--ignore':'.ignored_dir,.ignore_second',
'--diff': None,
'--clean': None
'--clean': None,
'--mode': None
}
)
with open(os.path.join(self.project_with_ignore_directory, "requirements.txt"), "r") as f:
@ -185,9 +234,27 @@ class TestPipreqs(unittest.TestCase):
for item in ['click', 'getpass']:
self.assertFalse(item.lower() in data)
def test_omit_version(self):
def test_dynamic_version_no_pin_scheme(self):
"""
Test --no-pin parameter
Test --mode=no-pin
"""
pipreqs.init(
{'<path>': self.project_with_ignore_directory, '--savepath': None,
'--print': False, '--use-local': None, '--force': True,
'--proxy': None, '--pypi-server': None,
'--diff': None,
'--clean': None,
'--mode': 'no-pin'
}
)
with open(os.path.join(self.project_with_ignore_directory, "requirements.txt"), "r") as f:
data = f.read().lower()
for item in ['beautifulsoup4', 'boto']:
self.assertTrue(item.lower() in data)
def test_dynamic_version_gt_scheme(self):
"""
Test --mode=gt
"""
pipreqs.init(
{'<path>': self.project_with_ignore_directory, '--savepath': None, '--print': False,
@ -196,19 +263,90 @@ class TestPipreqs(unittest.TestCase):
'--pypi-server': None,
'--diff': None,
'--clean': None,
'--no-pin': True
'--mode': 'gt'
}
)
with open(os.path.join(self.project_with_ignore_directory, "requirements.txt"), "r") as f:
data = f.readlines()
for item in data:
symbol = '>='
message = 'symbol is not in item'
self.assertIn(symbol, item, message)
def test_dynamic_version_compat_scheme(self):
"""
Test --mode=compat
"""
pipreqs.init(
{'<path>': self.project_with_ignore_directory, '--savepath': None, '--print': False,
'--use-local': None, '--force': True,
'--proxy': None,
'--pypi-server': None,
'--diff': None,
'--clean': None,
'--mode': 'compat'
}
)
with open(os.path.join(self.project_with_ignore_directory, "requirements.txt"), "r") as f:
data = f.readlines()
for item in data:
symbol = '~='
message = 'symbol is not in item'
self.assertIn(symbol, item, message)
def test_clean(self):
"""
Test --clean parameter
"""
pipreqs.init(
{'<path>': self.project, '--savepath': None, '--print': False,
'--use-local': None, '--force': True, '--proxy': None,
'--pypi-server': None, '--diff': None, '--clean': None,
'--mode': None}
)
assert os.path.exists(self.requirements_path) == 1
pipreqs.init(
{'<path>': self.project, '--savepath': None, '--print': False,
'--use-local': None, '--force': None, '--proxy': None,
'--pypi-server': None, '--diff': None,
'--clean': self.requirements_path, '--mode': 'non-pin'}
)
with open(self.requirements_path, "r") as f:
data = f.read().lower()
for item in ['beautifulsoup4==4.8.1', 'boto==2.49.0']:
self.assertFalse(item.lower() in data)
for item in self.modules[:-3]:
self.assertTrue(item.lower() in data)
def test_clean_with_imports_to_clean(self):
"""
Test --clean parameter when there are imports to clean
"""
cleaned_module = 'sqlalchemy'
pipreqs.init(
{'<path>': self.project, '--savepath': None, '--print': False,
'--use-local': None, '--force': True, '--proxy': None,
'--pypi-server': None, '--diff': None, '--clean': None,
'--mode': None}
)
assert os.path.exists(self.requirements_path) == 1
modules_clean = [m for m in self.modules if m != cleaned_module]
pipreqs.init(
{'<path>': self.project_clean, '--savepath': None,
'--print': False, '--use-local': None, '--force': None,
'--proxy': None, '--pypi-server': None, '--diff': None,
'--clean': self.requirements_path, '--mode': 'non-pin'}
)
with open(self.requirements_path, "r") as f:
data = f.read().lower()
self.assertTrue(cleaned_module not in data)
def test_import_notebooks(self):
"""
Test the function get_all_imports() using .ipynb file
"""
imports = pipreqs.get_all_imports(self.project_with_notebooks, encoding="utf-8")
imports = pipreqs.get_all_imports(
self.project_with_notebooks,
encoding="utf-8"
)
self.assertEqual(len(imports), 13)
for item in imports:
self.assertTrue(
@ -226,7 +364,11 @@ class TestPipreqs(unittest.TestCase):
"""
Test that invalid notebook files cannot be imported.
"""
self.assertRaises(SyntaxError, pipreqs.get_all_imports, self.project_with_invalid_notebooks)
self.assertRaises(
SyntaxError,
pipreqs.get_all_imports,
self.project_with_invalid_notebooks
)
def test_ipynb_2_py(self):
"""
@ -236,7 +378,10 @@ class TestPipreqs(unittest.TestCase):
parsed = pipreqs.get_all_imports(self.compatible_files_path["notebook"])
self.assertEqual(expected, parsed)
parsed = pipreqs.get_all_imports(self.compatible_files_path["notebook"], encoding="utf-8")
parsed = pipreqs.get_all_imports(
self.compatible_files_path["notebook"],
encoding="utf-8"
)
self.assertEqual(expected, parsed)
def test_filter_ext(self):

19
tox.ini
View File

@ -1,16 +1,17 @@
[tox]
envlist = py27, py34, py35, py36, pypy, flake8
envlist = py36, py37, py38, py39, pypy3, flake8
[gh-actions]
python =
3.6: py36
3.7: py37
3.8: py38
3.9: py39
pypy-3.7: pypy3
[testenv]
setenv =
PYTHONPATH = {toxinidir}:{toxinidir}/pipreqs
commands = python setup.py test
deps =
-r{toxinidir}/requirements.txt
[testenv:flake8]
basepython = python3.6
commands = flake8 pipreqs
deps =
-r{toxinidir}/requirements.txt
flake8
-r{toxinidir}/requirements.txt