ASCEE/pyqtgraph
5
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Expand CI + pre-commit (#991)

Browse Source 
* Initial attempt at extra checks in CI land

* Adding flake8 config

* Adding pre-commit configuration and explanation in CONTRIBUTING.md
This commit is contained in:
Ogi Moore 2019-08-29 13:56:25 -07:00 committed by GitHub
parent fd11e1352d
commit 584c4516f0
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
8 changed files with 469 additions and 202 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

49
.flake8 Normal file
View File

@ -0,0 +1,49 @@
[flake8]
exclude = .git,.tox,__pycache__,doc,old,build,dist
show_source = True
statistics = True
verbose = 2
select =
E101,
E112,
E122,
E125,
E133,
E223,
E224,
E242,
E273,
E274,
E901,
E902,
W191,
W601,
W602,
W603,
W604,
E124,
E231,
E211,
E261,
E271,
E272,
E304,
F401,
F402,
F403,
F404,
E501,
E502,
E702,
E703,
E711,
E712,
E721,
F811,
F812,
F821,
F822,
F823,
F831,
F841,
W292

11
.pre-commit-config.yaml Normal file
View File

@ -0,0 +1,11 @@
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
sha: master
hooks:
- id: check-added-large-files
args: ['--maxkb=100']
- id: check-case-conflict
- id: end-of-file-fixer
- id: fix-encoding-pragma
- id: mixed-line-ending
args: [--fix=lf]

16
CONTRIBUTING.md
View File

@ -1,6 +1,6 @@
# Contributing to PyQtGraph
Contributions to pyqtgraph are welcome!
Contributions to pyqtgraph are welcome!
Please use the following guidelines when preparing changes:
@ -13,11 +13,13 @@ Please use the following guidelines when preparing changes:
## Documentation
* Writing proper documentation and unit tests is highly encouraged. PyQtGraph uses nose / pytest style testing, so tests should usually be included in a tests/ directory adjacent to the relevant code.
* Writing proper documentation and unit tests is highly encouraged. PyQtGraph uses nose / pytest style testing, so tests should usually be included in a tests/ directory adjacent to the relevant code.
* Documentation is generated with sphinx; please check that docstring changes compile correctly
## Style guidelines
### Rules
* PyQtGraph prefers PEP8 for most style issues, but this is not enforced rigorously as long as the code is clean and readable.
* Use `python setup.py style` to see whether your code follows the mandatory style guidelines checked by flake8.
* Exception 1: All variable names should use camelCase rather than underscore_separation. This is done for consistency with Qt
@ -33,9 +35,15 @@ Please use the following guidelines when preparing changes:
============== ========================================================
```
QObject subclasses that implement new signals should also describe
QObject subclasses that implement new signals should also describe
these in a similar table.
### Pre-Commit
PyQtGraph developers are highly encouraged to (but not required) to use [`pre-commit`](https://pre-commit.com/). `pre-commit` does a number of checks when attempting to commit the code to ensure it conforms to various standards, such as `flake8`, utf-8 encoding pragma, line-ending fixers, and so on. If any of the checks fail, the commit will be rejected, and you will have the opportunity to make the necessary fixes before adding and committing a file again. This ensures that every commit made conforms to (most) of the styling standards that the library enforces; and you will most likely pass the code style checks by the CI.
To make use of `pre-commit`, have it available in your `$PATH` and run `pre-commit install` from the root directory of PyQtGraph.
## Testing Setting up a test environment
### Dependencies

82
azure-pipelines.yml
View File

@ -1,7 +1,3 @@
############################################################################################
# This config was rectrieved in no small part from https://github.com/slaclab/pydm
############################################################################################
trigger:
branches:
include:
@ -20,19 +16,83 @@ pr:
variables:
OFFICIAL_REPO: 'pyqtgraph/pyqtgraph'
DEFAULT_MERGE_BRANCH: 'develop'
jobs:
- template: azure-test-template.yml
parameters:
name: Linux
stages:
- stage: "pre_test"
jobs:
- job: check_diff_size
pool:
vmImage: 'Ubuntu 16.04'
steps:
- bash: |
git config --global advice.detachedHead false
mkdir ~/repo-clone && cd ~/repo-clone
git init
git remote add -t $(Build.SourceBranchName) origin $(Build.Repository.Uri)
git remote add -t ${DEFAULT_MERGE_BRANCH} upstream https://github.com/${OFFICIAL_REPO}.git
git fetch origin $(Build.SourceBranchName)
git fetch upstream ${DEFAULT_MERGE_BRANCH}
git checkout $(Build.SourceBranchName)
MERGE_SIZE=`du -s . | sed -e "s/\t.*//"`
echo -e "Merge Size ${MERGE_SIZE}"
git checkout ${DEFAULT_MERGE_BRANCH}
TARGET_SIZE=`du -s . | sed -e "s/\t.*//"`
echo -e "Target Size ${TARGET_SIZE}"
if [ "${MERGE_SIZE}" != "${TARGET_SIZE}" ]; then
SIZE_DIFF=`expr \( ${MERGE_SIZE} - ${TARGET_SIZE} \)`;
else
SIZE_DIFF=0;
fi;
echo -e "Estimated content size difference = ${SIZE_DIFF} kB" &&
test ${SIZE_DIFF} -lt 100;
displayName: 'Diff Size Check'
continueOnError: true
- job: "style_check"
pool:
vmImage: "Ubuntu 16.04"
steps:
- task: UsePythonVersion@0
inputs:
versionSpec: 3.7
- bash: |
pip install flake8
python setup.py style
displayName: 'flake8 check'
continueOnError: true
- job: "build_wheel"
pool:
vmImage: 'Ubuntu 16.04'
steps:
- task: UsePythonVersion@0
inputs:
versionSpec: 3.7
- script: |
python -m pip install setuptools wheel
python setup.py bdist_wheel --universal
displayName: "Build Python Wheel"
continueOnError: false
- publish: dist
artifact: wheel
- stage: "test"
jobs:
- template: azure-test-template.yml
parameters:
name: Windows
name: linux
vmImage: 'Ubuntu 16.04'
- template: azure-test-template.yml
parameters:
name: windows
vmImage: 'vs2017-win2016'
- template: azure-test-template.yml
parameters:
name: MacOS
name: macOS
vmImage: 'macOS-10.13'

93
azure-test-template.yml
View File

@ -26,16 +26,22 @@ jobs:
python.version: "3.6"
qt.bindings: "pyside2"
install.method: "conda"
Python37-PyQt-5.12:
Python37-PyQt-5.13:
python.version: '3.7'
qt.bindings: "PyQt5"
install.method: "pip"
Python37-PySide2-5.12:
Python37-PySide2-5.13:
python.version: "3.7"
qt.bindings: "PySide2"
install.method: "pip"
steps:
- task: DownloadPipelineArtifact@2
inputs:
source: 'current'
artifact: wheel
path: 'dist'
- task: ScreenResolutionUtility@1
inputs:
displaySettings: 'specific'
@ -43,6 +49,11 @@ jobs:
height: '1080'
condition: eq(variables['agent.os'], 'Windows_NT' )
- task: UsePythonVersion@0
inputs:
versionSpec: $(python.version)
condition: eq(variables['install.method'], 'pip')
- script: |
curl -LJO https://github.com/pal1000/mesa-dist-win/releases/download/19.1.0/mesa3d-19.1.0-release-msvc.exe
7z x mesa3d-19.1.0-release-msvc.exe
@ -60,75 +71,71 @@ jobs:
displayName: "Install Windows-Mesa OpenGL DLL"
condition: eq(variables['agent.os'], 'Windows_NT')
- task: UsePythonVersion@0
inputs:
versionSpec: $(python.version)
condition: eq(variables['install.method'], 'pip')
- bash: |
if [ $(agent.os) == 'Linux' ]
then
echo '##vso[task.prependpath]/usr/share/miniconda/bin'
echo "##vso[task.prependpath]$CONDA/bin"
if [ $(python.version) == '2.7' ]
then
echo "Grabbing Older Miniconda"
wget https://repo.anaconda.com/miniconda/Miniconda2-4.6.14-Linux-x86_64.sh -O Miniconda.sh
bash Miniconda.sh -b -p $CONDA -f
fi
elif [ $(agent.os) == 'Darwin' ]
then
echo '##vso[task.prependpath]$CONDA/bin'
sudo install -d -m 0777 /usr/local/miniconda/envs
sudo chown -R $USER $CONDA
echo "##vso[task.prependpath]$CONDA/bin"
if [ $(python.version) == '2.7' ]
then
echo "Grabbing Older Miniconda"
wget https://repo.anaconda.com/miniconda/Miniconda2-4.6.14-MacOSX-x86_64.sh -O Miniconda.sh
bash Miniconda.sh -b -p $CONDA -f
fi
elif [ $(agent.os) == 'Windows_NT' ]
then
echo "##vso[task.prependpath]$env:CONDA\Scripts"
echo "##vso[task.prependpath]$CONDA/Scripts"
else
echo 'Just what OS are you using?'
fi
displayName: 'Add Conda to $PATH'
displayName: 'Add Conda To $PATH'
condition: eq(variables['install.method'], 'conda' )
- task: CondaEnvironment@0
displayName: 'Create Conda Environment'
condition: eq(variables['install.method'], 'conda')
inputs:
environmentName: 'test-environment-$(python.version)'
packageSpecs: 'python=$(python.version)'
updateConda: false
- bash: |
if [ $(install.method) == "conda" ]
then
conda create --name test-environment-$(python.version) python=$(python.version) --yes
echo "Conda Info:"
conda info
echo "Installing qt-bindings"
source activate test-environment-$(python.version)
conda install -c conda-forge $(qt.bindings) numpy scipy pyopengl pytest six coverage --yes --quiet
if [ $(agent.os) == "Linux" ] && [ $(python.version) == "2.7" ]
then
conda install $(qt.bindings) --yes
else
conda install -c conda-forge $(qt.bindings) --yes
fi
echo "Installing remainder of dependencies"
conda install -c conda-forge numpy scipy six pyopengl --yes
else
pip install $(qt.bindings) numpy scipy pyopengl pytest six coverage
pip install $(qt.bindings) numpy scipy pyopengl six
fi
pip install pytest-xdist pytest-cov
echo ""
pip install pytest pytest-xdist pytest-cov coverage
if [ $(python.version) == "2.7" ]
then
pip install pytest-faulthandler
pip install pytest-faulthandler==1.6.0
export PYTEST_ADDOPTS="--faulthandler-timeout=15"
fi
displayName: "Install Dependencies"
- bash: |
if [ $(install.method) == "conda" ]
then
source activate test-environment-$(python.version)
fi
pip install setuptools wheel
python setup.py bdist_wheel
pip install dist/*.whl
displayName: 'Build Wheel and Install'
- task: CopyFiles@2
inputs:
contents: 'dist/**'
targetFolder: $(Build.ArtifactStagingDirectory)
cleanTargetFolder: true
displayName: "Copy Binary Wheel Distribution To Artifacts"
- task: PublishBuildArtifacts@1
displayName: 'Publish Binary Wheel'
condition: always()
inputs:
pathtoPublish: $(Build.ArtifactStagingDirectory)/dist
artifactName: Distributions
python -m pip install --no-index --find-links=dist pyqtgraph
displayName: 'Install Wheel'
- bash: |
sudo apt-get install -y libxkbcommon-x11-0 # herbstluftwm

157
examples/test_examples.py
View File

@ -1,3 +1,4 @@
# -*- coding: utf-8 -*-
from __future__ import print_function, division, absolute_import
from pyqtgraph import Qt
from . import utils
@ -5,7 +6,6 @@ from collections import namedtuple
import errno
import importlib
import itertools
import pkgutil
import pytest
import os, sys
import subprocess
@ -41,7 +41,12 @@ if os.getenv('TRAVIS') is not None:
files = sorted(set(utils.buildFileList(utils.examples)))
frontends = {Qt.PYQT4: False, Qt.PYQT5: False, Qt.PYSIDE: False, Qt.PYSIDE2: False}
frontends = {
Qt.PYQT4: False,
Qt.PYQT5: False,
Qt.PYSIDE: False,
Qt.PYSIDE2: False
}
# sort out which of the front ends are available
for frontend in frontends.keys():
try:
@ -50,48 +55,136 @@ for frontend in frontends.keys():
except ImportError:
pass
installedFrontends = sorted([frontend for frontend, isPresent in frontends.items() if isPresent])
installedFrontends = sorted([
frontend for frontend, isPresent in frontends.items() if isPresent
])
exceptionCondition = namedtuple("exceptionCondition", ["condition", "reason"])
conditionalExampleTests = {
"hdf5.py": exceptionCondition(False, reason="Example requires user interaction and is not suitable for testing"),
"RemoteSpeedTest.py": exceptionCondition(False, reason="Test is being problematic on CI machines"),
"optics_demos.py": exceptionCondition(not frontends[Qt.PYSIDE], reason="Test fails due to PySide bug: https://bugreports.qt.io/browse/PYSIDE-671"),
'GLVolumeItem.py': exceptionCondition(not(sys.platform == "darwin" and sys.version_info[0] == 2 and (frontends[Qt.PYQT4] or frontends[Qt.PYSIDE])), reason="glClear does not work on macOS + Python2.7 + Qt4: https://github.com/pyqtgraph/pyqtgraph/issues/939"),
'GLIsosurface.py': exceptionCondition(not(sys.platform == "darwin" and sys.version_info[0] == 2 and (frontends[Qt.PYQT4] or frontends[Qt.PYSIDE])), reason="glClear does not work on macOS + Python2.7 + Qt4: https://github.com/pyqtgraph/pyqtgraph/issues/939"),
'GLSurfacePlot.py': exceptionCondition(not(sys.platform == "darwin" and sys.version_info[0] == 2 and (frontends[Qt.PYQT4] or frontends[Qt.PYSIDE])), reason="glClear does not work on macOS + Python2.7 + Qt4: https://github.com/pyqtgraph/pyqtgraph/issues/939"),
'GLScatterPlotItem.py': exceptionCondition(not(sys.platform == "darwin" and sys.version_info[0] == 2 and (frontends[Qt.PYQT4] or frontends[Qt.PYSIDE])), reason="glClear does not work on macOS + Python2.7 + Qt4: https://github.com/pyqtgraph/pyqtgraph/issues/939"),
'GLshaders.py': exceptionCondition(not(sys.platform == "darwin" and sys.version_info[0] == 2 and (frontends[Qt.PYQT4] or frontends[Qt.PYSIDE])), reason="glClear does not work on macOS + Python2.7 + Qt4: https://github.com/pyqtgraph/pyqtgraph/issues/939"),
'GLLinePlotItem.py': exceptionCondition(not(sys.platform == "darwin" and sys.version_info[0] == 2 and (frontends[Qt.PYQT4] or frontends[Qt.PYSIDE])), reason="glClear does not work on macOS + Python2.7 + Qt4: https://github.com/pyqtgraph/pyqtgraph/issues/939"),
'GLMeshItem.py': exceptionCondition(not(sys.platform == "darwin" and sys.version_info[0] == 2 and (frontends[Qt.PYQT4] or frontends[Qt.PYSIDE])), reason="glClear does not work on macOS + Python2.7 + Qt4: https://github.com/pyqtgraph/pyqtgraph/issues/939"),
'GLImageItem.py': exceptionCondition(not(sys.platform == "darwin" and sys.version_info[0] == 2 and (frontends[Qt.PYQT4] or frontends[Qt.PYSIDE])), reason="glClear does not work on macOS + Python2.7 + Qt4: https://github.com/pyqtgraph/pyqtgraph/issues/939")
conditionalExamples = {
"hdf5.py": exceptionCondition(
False,
reason="Example requires user interaction"
),
"RemoteSpeedTest.py": exceptionCondition(
False,
reason="Test is being problematic on CI machines"
),
"optics_demos.py": exceptionCondition(
not frontends[Qt.PYSIDE],
reason=(
"Test fails due to PySide bug: ",
"https://bugreports.qt.io/browse/PYSIDE-671"
)
),
'GLVolumeItem.py': exceptionCondition(
not(sys.platform == "darwin" and
sys.version_info[0] == 2 and
(frontends[Qt.PYQT4] or frontends[Qt.PYSIDE])),
reason=(
"glClear does not work on macOS + Python2.7 + Qt4: ",
"https://github.com/pyqtgraph/pyqtgraph/issues/939"
)
),
'GLIsosurface.py': exceptionCondition(
not(sys.platform == "darwin" and
sys.version_info[0] == 2 and
(frontends[Qt.PYQT4] or frontends[Qt.PYSIDE])),
reason=(
"glClear does not work on macOS + Python2.7 + Qt4: ",
"https://github.com/pyqtgraph/pyqtgraph/issues/939"
)
),
'GLSurfacePlot.py': exceptionCondition(
not(sys.platform == "darwin" and
sys.version_info[0] == 2 and
(frontends[Qt.PYQT4] or frontends[Qt.PYSIDE])),
reason=(
"glClear does not work on macOS + Python2.7 + Qt4: ",
"https://github.com/pyqtgraph/pyqtgraph/issues/939"
)
),
'GLScatterPlotItem.py': exceptionCondition(
not(sys.platform == "darwin" and
sys.version_info[0] == 2 and
(frontends[Qt.PYQT4] or frontends[Qt.PYSIDE])),
reason=(
"glClear does not work on macOS + Python2.7 + Qt4: ",
"https://github.com/pyqtgraph/pyqtgraph/issues/939"
)
),
'GLshaders.py': exceptionCondition(
not(sys.platform == "darwin" and
sys.version_info[0] == 2 and
(frontends[Qt.PYQT4] or frontends[Qt.PYSIDE])),
reason=(
"glClear does not work on macOS + Python2.7 + Qt4: ",
"https://github.com/pyqtgraph/pyqtgraph/issues/939"
)
),
'GLLinePlotItem.py': exceptionCondition(
not(sys.platform == "darwin" and
sys.version_info[0] == 2 and
(frontends[Qt.PYQT4] or frontends[Qt.PYSIDE])),
reason=(
"glClear does not work on macOS + Python2.7 + Qt4: ",
"https://github.com/pyqtgraph/pyqtgraph/issues/939"
)
),
'GLMeshItem.py': exceptionCondition(
not(sys.platform == "darwin" and
sys.version_info[0] == 2 and
(frontends[Qt.PYQT4] or frontends[Qt.PYSIDE])),
reason=(
"glClear does not work on macOS + Python2.7 + Qt4: ",
"https://github.com/pyqtgraph/pyqtgraph/issues/939"
)
),
'GLImageItem.py': exceptionCondition(
not(sys.platform == "darwin" and
sys.version_info[0] == 2 and
(frontends[Qt.PYQT4] or frontends[Qt.PYSIDE])),
reason=(
"glClear does not work on macOS + Python2.7 + Qt4: ",
"https://github.com/pyqtgraph/pyqtgraph/issues/939"
)
)
}
@pytest.mark.parametrize(
"frontend, f",
[
pytest.param(
frontend,
"frontend, f",
[
pytest.param(
frontend,
f,
marks=pytest.mark.skipif(conditionalExampleTests[f[1]].condition is False,
reason=conditionalExampleTests[f[1]].reason) if f[1] in conditionalExampleTests.keys() else (),
)
for frontend, f, in itertools.product(installedFrontends, files)
],
ids = [" {} - {} ".format(f[1], frontend) for frontend, f in itertools.product(installedFrontends, files)]
marks=pytest.mark.skipif(
conditionalExamples[f[1]].condition is False,
reason=conditionalExamples[f[1]].reason
) if f[1] in conditionalExamples.keys() else (),
)
for frontend, f, in itertools.product(installedFrontends, files)
],
ids = [
" {} - {} ".format(f[1], frontend)
for frontend, f in itertools.product(
installedFrontends,
files
)
]
)
def testExamples(frontend, f, graphicsSystem=None):
# runExampleFile(f[0], f[1], sys.executable, frontend)
name, file = f
global path
fn = os.path.join(path,file)
fn = os.path.join(path, file)
os.chdir(path)
sys.stdout.write("{} ".format(name))
sys.stdout.flush()
import1 = "import %s" % frontend if frontend != '' else ''
import2 = os.path.splitext(os.path.split(fn)[1])[0]
graphicsSystem = '' if graphicsSystem is None else "pg.QtGui.QApplication.setGraphicsSystem('%s')" % graphicsSystem
graphicsSystem = (
'' if graphicsSystem is None else "pg.QtGui.QApplication.setGraphicsSystem('%s')" % graphicsSystem
)
code = """
try:
%s
@ -123,7 +216,7 @@ except:
stderr=subprocess.PIPE,
stdout=subprocess.PIPE)
process.stdin.write(code.encode('UTF-8'))
process.stdin.close() ##?
process.stdin.close()
output = ''
fail = False
while True:
@ -146,10 +239,14 @@ except:
process.kill()
#res = process.communicate()
res = (process.stdout.read(), process.stderr.read())
if fail or 'exception' in res[1].decode().lower() or 'error' in res[1].decode().lower():
if (fail or
'exception' in res[1].decode().lower() or
'error' in res[1].decode().lower()):
print(res[0].decode())
print(res[1].decode())
pytest.fail("{}\n{}\nFailed {} Example Test Located in {} ".format(res[0].decode(), res[1].decode(), name, file), pytrace=False)
pytest.fail("{}\n{}\nFailed {} Example Test Located in {} "
.format(res[0].decode(), res[1].decode(), name, file),
pytrace=False)
if __name__ == "__main__":
pytest.cmdline.main()

7
setup.py
View File

@ -1,5 +1,6 @@
# -*- coding: utf-8 -*-
DESCRIPTION = """\
PyQtGraph is a pure-python graphics and GUI library built on PyQt4/PySide and
PyQtGraph is a pure-python graphics and GUI library built on PyQt4/PyQt5/PySide/PySide2 and
numpy.
It is intended for use in mathematics / scientific / engineering applications.
@ -12,14 +13,13 @@ setupOpts = dict(
name='pyqtgraph',
description='Scientific Graphics and GUI Library for Python',
long_description=DESCRIPTION,
license='MIT',
license = 'MIT',
url='http://www.pyqtgraph.org',
author='Luke Campagnola',
author_email='luke.campagnola@gmail.com',
classifiers = [
"Programming Language :: Python",
"Programming Language :: Python :: 2",
"Programming Language :: Python :: 2.6",
"Programming Language :: Python :: 2.7",
"Programming Language :: Python :: 3",
"Development Status :: 4 - Beta",
@ -145,4 +145,3 @@ setup(
],
**setupOpts
)

256
tools/setupHelpers.py
View File

@ -10,14 +10,15 @@ except ImportError:
output = proc.stdout.read()
proc.wait()
if proc.returncode != 0:
ex = Exception("Process had nonzero return value %d" % proc.returncode)
ex = Exception("Process had nonzero return value "
+ "%d " % proc.returncode)
ex.returncode = proc.returncode
ex.output = output
raise ex
return output
# Maximum allowed repository size difference (in kB) following merge.
# This is used to prevent large files from being inappropriately added to
# This is used to prevent large files from being inappropriately added to
# the repository history.
MERGE_SIZE_LIMIT = 100
@ -42,19 +43,19 @@ FLAKE_MANDATORY = set([
'E901', # SyntaxError or IndentationError
'E902', # IOError
'W191', # indentation contains tabs
'W601', # .has_key() is deprecated, use ‘in’
'W602', # deprecated form of raising exception
'W603', # ‘<>’ is deprecated, use ‘!=’
'W604', # backticks are deprecated, use ‘repr()’
'W604', # backticks are deprecated, use ‘repr()’
])
FLAKE_RECOMMENDED = set([
'E124', # closing bracket does not match visual indentation
'E231', # missing whitespace after ‘,’
'E211', # whitespace before ‘(‘
'E261', # at least two spaces before inline comment
'E271', # multiple spaces after keyword
@ -65,10 +66,10 @@ FLAKE_RECOMMENDED = set([
'F402', # import module from line N shadowed by loop variable
'F403', # ‘from module import *’ used; unable to detect undefined names
'F404', # future import(s) name after other statements
'E501', # line too long (82 > 79 characters)
'E502', # the backslash is redundant between brackets
'E702', # multiple statements on one line (semicolon)
'E703', # statement ends with a semicolon
'E711', # comparison to None should be ‘if cond is None:’
@ -82,7 +83,7 @@ FLAKE_RECOMMENDED = set([
'F823', # local variable name ... referenced before assignment
'F831', # duplicate argument name in function definition
'F841', # local variable name is assigned to but never used
'W292', # no newline at end of file
])
@ -93,7 +94,7 @@ FLAKE_OPTIONAL = set([
'E126', # continuation line over-indented for hanging indent
'E127', # continuation line over-indented for visual indent
'E128', # continuation line under-indented for visual indent
'E201', # whitespace after ‘(‘
'E202', # whitespace before ‘)’
'E203', # whitespace before ‘:’
@ -105,19 +106,19 @@ FLAKE_OPTIONAL = set([
'E228', # missing whitespace around modulo operator
'E241', # multiple spaces after ‘,’
'E251', # unexpected spaces around keyword / parameter equals
'E262', # inline comment should start with ‘# ‘
'E262', # inline comment should start with ‘# ‘
'E301', # expected 1 blank line, found 0
'E302', # expected 2 blank lines, found 0
'E303', # too many blank lines (3)
'E401', # multiple imports on one line
'E701', # multiple statements on one line (colon)
'W291', # trailing whitespace
'W293', # blank line contains whitespace
'W391', # blank line at end of file
])
@ -128,23 +129,10 @@ FLAKE_IGNORE = set([
])
#def checkStyle():
#try:
#out = check_output(['flake8', '--select=%s' % FLAKE_TESTS, '--statistics', 'pyqtgraph/'])
#ret = 0
#print("All style checks OK.")
#except Exception as e:
#out = e.output
#ret = e.returncode
#print(out.decode('utf-8'))
#return ret
def checkStyle():
""" Run flake8, checking only lines that are modified since the last
git commit. """
test = [ 1,2,3 ]
# First check _all_ code against mandatory error codes
print('flake8: check all code against mandatory error set...')
errors = ','.join(FLAKE_MANDATORY)
@ -154,39 +142,47 @@ def checkStyle():
output = proc.stdout.read().decode('utf-8')
ret = proc.wait()
printFlakeOutput(output)
# Check for DOS newlines
print('check line endings in all files...')
count = 0
allowedEndings = set([None, '\n'])
for path, dirs, files in os.walk('.'):
if path.startswith("." + os.path.sep + ".tox"):
continue
for f in files:
if os.path.splitext(f)[1] not in ('.py', '.rst'):
continue
filename = os.path.join(path, f)
fh = open(filename, 'U')
x = fh.readlines()
endings = set(fh.newlines if isinstance(fh.newlines, tuple) else (fh.newlines,))
_ = fh.readlines()
endings = set(
fh.newlines
if isinstance(fh.newlines, tuple)
else (fh.newlines,)
)
endings -= allowedEndings
if len(endings) > 0:
print("\033[0;31m" + "File has invalid line endings: %s" % filename + "\033[0m")
print("\033[0;31m"
+ "File has invalid line endings: "
+ "%s" % filename + "\033[0m")
ret = ret | 2
count += 1
print('checked line endings in %d files' % count)
# Next check new code with optional error codes
print('flake8: check new code against recommended error set...')
diff = subprocess.check_output(['git', 'diff'])
proc = subprocess.Popen(['flake8', '--diff', #'--show-source',
proc = subprocess.Popen(['flake8', '--diff', # '--show-source',
'--ignore=' + errors],
stdin=subprocess.PIPE,
stdin=subprocess.PIPE,
stdout=subprocess.PIPE)
proc.stdin.write(diff)
proc.stdin.close()
output = proc.stdout.read().decode('utf-8')
ret |= printFlakeOutput(output)
if ret == 0:
print('style test passed.')
else:
@ -244,14 +240,20 @@ def unitTests():
return ret
def checkMergeSize(sourceBranch=None, targetBranch=None, sourceRepo=None, targetRepo=None):
def checkMergeSize(
sourceBranch=None,
targetBranch=None,
sourceRepo=None,
targetRepo=None
):
"""
Check that a git merge would not increase the repository size by MERGE_SIZE_LIMIT.
Check that a git merge would not increase the repository size by
MERGE_SIZE_LIMIT.
"""
if sourceBranch is None:
sourceBranch = getGitBranch()
sourceRepo = '..'
if targetBranch is None:
if sourceBranch == 'develop':
targetBranch = 'develop'
@ -259,38 +261,38 @@ def checkMergeSize(sourceBranch=None, targetBranch=None, sourceRepo=None, target
else:
targetBranch = 'develop'
targetRepo = '..'
workingDir = '__merge-test-clone'
env = dict(TARGET_BRANCH=targetBranch,
SOURCE_BRANCH=sourceBranch,
TARGET_REPO=targetRepo,
env = dict(TARGET_BRANCH=targetBranch,
SOURCE_BRANCH=sourceBranch,
TARGET_REPO=targetRepo,
SOURCE_REPO=sourceRepo,
WORKING_DIR=workingDir,
)
print("Testing merge size difference:\n"
" SOURCE: {SOURCE_REPO} {SOURCE_BRANCH}\n"
" TARGET: {TARGET_BRANCH} {TARGET_REPO}".format(**env))
setup = """
mkdir {WORKING_DIR} && cd {WORKING_DIR} &&
git init && git remote add -t {TARGET_BRANCH} target {TARGET_REPO} &&
git fetch target {TARGET_BRANCH} &&
git checkout -qf target/{TARGET_BRANCH} &&
git fetch target {TARGET_BRANCH} &&
git checkout -qf target/{TARGET_BRANCH} &&
git gc -q --aggressive
""".format(**env)
checkSize = """
cd {WORKING_DIR} &&
cd {WORKING_DIR} &&
du -s . | sed -e "s/\t.*//"
""".format(**env)
merge = """
cd {WORKING_DIR} &&
git pull -q {SOURCE_REPO} {SOURCE_BRANCH} &&
git pull -q {SOURCE_REPO} {SOURCE_BRANCH} &&
git gc -q --aggressive
""".format(**env)
try:
print("Check out target branch:\n" + setup)
check_call(setup, shell=True)
@ -300,13 +302,17 @@ def checkMergeSize(sourceBranch=None, targetBranch=None, sourceRepo=None, target
check_call(merge, shell=True)
mergeSize = int(check_output(checkSize, shell=True))
print("MERGE SIZE: %d kB" % mergeSize)
diff = mergeSize - targetSize
if diff <= MERGE_SIZE_LIMIT:
print("DIFFERENCE: %d kB [OK]" % diff)
return 0
else:
print("\033[0;31m" + "DIFFERENCE: %d kB [exceeds %d kB]" % (diff, MERGE_SIZE_LIMIT) + "\033[0m")
print("\033[0;31m"
+ "DIFFERENCE: %d kB [exceeds %d kB]" % (
diff,
MERGE_SIZE_LIMIT)
+ "\033[0m")
return 2
finally:
if os.path.isdir(workingDir):
@ -327,7 +333,11 @@ def mergeTests():
def listAllPackages(pkgroot):
path = os.getcwd()
n = len(path.split(os.path.sep))
subdirs = [i[0].split(os.path.sep)[n:] for i in os.walk(os.path.join(path, pkgroot)) if '__init__.py' in i[2]]
subdirs = [
i[0].split(os.path.sep)[n:]
for i in os.walk(os.path.join(path, pkgroot))
if '__init__.py' in i[2]
]
return ['.'.join(p) for p in subdirs]
@ -338,48 +348,61 @@ def getInitVersion(pkgroot):
init = open(initfile).read()
m = re.search(r'__version__ = (\S+)\n', init)
if m is None or len(m.groups()) != 1:
raise Exception("Cannot determine __version__ from init file: '%s'!" % initfile)
raise Exception("Cannot determine __version__ from init file: "
+ "'%s'!" % initfile)
version = m.group(1).strip('\'\"')
return version
def gitCommit(name):
"""Return the commit ID for the given name."""
commit = check_output(['git', 'show', name], universal_newlines=True).split('\n')[0]
commit = check_output(
['git', 'show', name],
universal_newlines=True).split('\n')[0]
assert commit[:7] == 'commit '
return commit[7:]
def getGitVersion(tagPrefix):
"""Return a version string with information about this git checkout.
If the checkout is an unmodified, tagged commit, then return the tag version.
If this is not a tagged commit, return the output of ``git describe --tags``.
If the checkout is an unmodified, tagged commit, then return the tag
version
If this is not a tagged commit, return the output of
``git describe --tags``
If this checkout has been modified, append "+" to the version.
"""
path = os.getcwd()
if not os.path.isdir(os.path.join(path, '.git')):
return None
v = check_output(['git', 'describe', '--tags', '--dirty', '--match=%s*'%tagPrefix]).strip().decode('utf-8')
v = check_output(['git',
'describe',
'--tags',
'--dirty',
'--match=%s*'%tagPrefix]).strip().decode('utf-8')
# chop off prefix
assert v.startswith(tagPrefix)
v = v[len(tagPrefix):]
# split up version parts
parts = v.split('-')
# has working tree been modified?
modified = False
if parts[-1] == 'dirty':
modified = True
parts = parts[:-1]
# have commits been added on top of last tagged version?
# (git describe adds -NNN-gXXXXXXX if this is the case)
local = None
if len(parts) > 2 and re.match(r'\d+', parts[-2]) and re.match(r'g[0-9a-f]{7}', parts[-1]):
if (len(parts) > 2 and
re.match(r'\d+', parts[-2]) and
re.match(r'g[0-9a-f]{7}', parts[-1])):
local = parts[-1]
parts = parts[:-2]
gitVersion = '-'.join(parts)
if local is not None:
gitVersion += '+' + local
@ -389,7 +412,10 @@ def getGitVersion(tagPrefix):
return gitVersion
def getGitBranch():
m = re.search(r'\* (.*)', check_output(['git', 'branch'], universal_newlines=True))
m = re.search(
r'\* (.*)',
check_output(['git', 'branch'],
universal_newlines=True))
if m is None:
return ''
else:
@ -397,32 +423,33 @@ def getGitBranch():
def getVersionStrings(pkg):
"""
Returns 4 version strings:
Returns 4 version strings:
* the version string to use for this build,
* version string requested with --force-version (or None)
* version string that describes the current git checkout (or None).
* version string in the pkg/__init__.py,
* version string in the pkg/__init__.py,
The first return value is (forceVersion or gitVersion or initVersion).
"""
## Determine current version string from __init__.py
initVersion = getInitVersion(pkgroot=pkg)
## If this is a git checkout, try to generate a more descriptive version string
# If this is a git checkout
# try to generate a more descriptive version string
try:
gitVersion = getGitVersion(tagPrefix=pkg+'-')
except:
gitVersion = None
sys.stderr.write("This appears to be a git checkout, but an error occurred "
"while attempting to determine a version string for the "
"current commit.\n")
sys.stderr.write("This appears to be a git checkout, but an error "
"occurred while attempting to determine a version "
"string for the current commit.\n")
sys.excepthook(*sys.exc_info())
# See whether a --force-version flag was given
forcedVersion = None
for i,arg in enumerate(sys.argv):
for i, arg in enumerate(sys.argv):
if arg.startswith('--force-version'):
if arg == '--force-version':
forcedVersion = sys.argv[i+1]
@ -431,8 +458,8 @@ def getVersionStrings(pkg):
elif arg.startswith('--force-version='):
forcedVersion = sys.argv[i].replace('--force-version=', '')
sys.argv.pop(i)
## Finally decide on a version string to use:
if forcedVersion is not None:
version = forcedVersion
@ -443,7 +470,8 @@ def getVersionStrings(pkg):
_, local = gitVersion.split('+')
if local != '':
version = version + '+' + local
sys.stderr.write("Detected git commit; will use version string: '%s'\n" % version)
sys.stderr.write("Detected git commit; "
+ "will use version string: '%s'\n" % version)
return version, forcedVersion, gitVersion, initVersion
@ -457,29 +485,31 @@ class DebCommand(Command):
maintainer = "Luke Campagnola <luke.campagnola@gmail.com>"
debTemplate = "debian"
debDir = "deb_build"
user_options = []
def initialize_options(self):
self.cwd = None
def finalize_options(self):
self.cwd = os.getcwd()
def run(self):
version = self.distribution.get_version()
pkgName = self.distribution.get_name()
debName = "python-" + pkgName
debDir = self.debDir
assert os.getcwd() == self.cwd, 'Must be in package root: %s' % self.cwd
assert os.getcwd() == self.cwd, 'Must be in package root: '
+ '%s' % self.cwd
if os.path.isdir(debDir):
raise Exception('DEB build dir already exists: "%s"' % debDir)
sdist = "dist/%s-%s.tar.gz" % (pkgName, version)
if not os.path.isfile(sdist):
raise Exception("No source distribution; run `setup.py sdist` first.")
raise Exception("No source distribution; "
+ "run `setup.py sdist` first.")
# copy sdist to build directory and extract
os.mkdir(debDir)
renamedSdist = '%s_%s.orig.tar.gz' % (debName, version)
@ -489,16 +519,20 @@ class DebCommand(Command):
if os.system("cd %s; tar -xzf %s" % (debDir, renamedSdist)) != 0:
raise Exception("Error extracting source distribution.")
buildDir = '%s/%s-%s' % (debDir, pkgName, version)
# copy debian control structure
print("copytree %s => %s" % (self.debTemplate, buildDir+'/debian'))
shutil.copytree(self.debTemplate, buildDir+'/debian')
# Write new changelog
chlog = generateDebianChangelog(pkgName, 'CHANGELOG', version, self.maintainer)
chlog = generateDebianChangelog(
pkgName,
'CHANGELOG',
version,
self.maintainer)
print("write changelog %s" % buildDir+'/debian/changelog')
open(buildDir+'/debian/changelog', 'w').write(chlog)
# build package
print('cd %s; debuild -us -uc' % buildDir)
if os.system('cd %s; debuild -us -uc' % buildDir) != 0:
@ -521,43 +555,45 @@ class DebugCommand(Command):
class TestCommand(Command):
description = "Run all package tests and exit immediately with informative return code."
description = "Run all package tests and exit immediately with ", \
"informative return code."
user_options = []
def run(self):
sys.exit(unitTests())
def initialize_options(self):
pass
def finalize_options(self):
pass
class StyleCommand(Command):
description = "Check all code for style, exit immediately with informative return code."
description = "Check all code for style, exit immediately with ", \
"informative return code."
user_options = []
def run(self):
sys.exit(checkStyle())
def initialize_options(self):
pass
def finalize_options(self):
pass
class MergeTestCommand(Command):
description = "Run all tests needed to determine whether the current code is suitable for merge."
description = "Run all tests needed to determine whether the current ",\
"code is suitable for merge."
user_options = []
def run(self):
sys.exit(mergeTests())
def initialize_options(self):
pass
def finalize_options(self):
pass