Skip to content

H
heroku-buildpack-python
Commit 47c40ce0 authored by Kenneth Reitz's avatar Kenneth Reitz
Browse files
Options

Squashed commit of smart-requirements branch 

commit 60a2a9d0720693b9438b0f1c75f570dc96bf3a4b
Author: Kenneth Reitz <me@kennethreitz.com>
Date:   Fri Aug 1 14:58:29 2014 -0400

    silent mode

commit 20368e741993a2c96739aa1de457618a5272bfe7
Author: Kenneth Reitz <me@kennethreitz.com>
Date:   Fri Aug 1 14:56:00 2014 -0400

    pylibmc

commit 8a512995d806ad58e4a8cab9cd712e762277dcf3
Author: Kenneth Reitz <me@kennethreitz.com>
Date:   Fri Aug 1 14:55:25 2014 -0400

    fix for pip-grep

commit 0b83a0ac11ee67f08dc843b01ecfe72c9b9c2938
Author: Kenneth Reitz <me@kennethreitz.com>
Date:   Fri Aug 1 14:50:35 2014 -0400

    cat that shit

commit 27095d320f4e6c0a6bccec945efbdb6f09598cfc
Author: Kenneth Reitz <me@kennethreitz.com>
Date:   Fri Aug 1 14:49:11 2014 -0400

    don't silence the output

commit db15ac279169467e1e78cfcc55cb740660d4b4c9
Author: Kenneth Reitz <me@kennethreitz.com>
Date:   Fri Aug 1 14:46:55 2014 -0400

    hello

commit 6b851815b07331bf185f218c40d62123df048b7b
Author: Kenneth Reitz <me@kennethreitz.com>
Date:   Fri Aug 1 14:41:37 2014 -0400

    cleanup

commit b2ccf76ccb244976e5ee71164cfe591925cb13a4
Author: Kenneth Reitz <me@kennethreitz.com>
Date:   Fri Aug 1 14:41:14 2014 -0400

    remove last debugging statement

commit fdcb7a210327f01fdc49dba00c3928cbb4778c72
Author: Kenneth Reitz <me@kennethreitz.com>
Date:   Fri Aug 1 14:39:10 2014 -0400

    remove debugging statements

commit dd58a898e4e5244f3b78ce0402537a1b730a3cc3
Author: Kenneth Reitz <me@kennethreitz.com>
Date:   Fri Aug 1 14:36:02 2014 -0400

    bp watch

commit c722f6e964b2a25f91d153c65f0c473ff22ff4c8
Author: Kenneth Reitz <me@kennethreitz.com>
Date:   Fri Aug 1 14:34:34 2014 -0400

    install finished

commit 81644ba1956798c06f55a8ce4c86239ca4ee8036
Author: Kenneth Reitz <me@kennethreitz.com>
Date:   Fri Aug 1 14:33:04 2014 -0400

    install started

commit 62d0e3b81a5ac25e8ceab735e25cf2bb8d161f0b
Author: Kenneth Reitz <me@kennethreitz.com>
Date:   Thu Jul 31 14:54:59 2014 -0400

    debug

commit 1775a1b206780fd99d013ca7c0cb8d8de17a8461
Author: Kenneth Reitz <me@kennethreitz.com>
Date:   Thu Jul 31 13:01:15 2014 -0400

    test0

commit 3d20cf89ae0f9436a54c456779a544a3b9d226ac
Author: Kenneth Reitz <me@kennethreitz.com>
Date:   Thu Jul 31 12:57:07 2014 -0400

    test2/2

commit 751d7496893bc84430a93cd479511786bce9abb9
Author: Kenneth Reitz <me@kennethreitz.com>
Date:   Thu Jul 31 12:50:11 2014 -0400

    silent mode

commit f927b7b3dee18c6ee58b36a3f96ad66e53b8d46d
Author: Kenneth Reitz <me@kennethreitz.com>
Date:   Thu Jul 31 12:44:58 2014 -0400

    pylibmc with pip-grep

commit 00dfc36701bf72191f0ad7b7d5533ba7a9ac3b10
Author: Kenneth Reitz <me@kennethreitz.com>
Date:   Thu Jul 31 12:41:11 2014 -0400

    no debug echo

commit 38d78d800583fc89100cadc551ce715c5800f0b8
Author: Kenneth Reitz <me@kennethreitz.com>
Date:   Thu Jul 31 12:40:06 2014 -0400

    source, not subenv

commit f455de19a8c1150aa2a4df164b90c2dc9a195942
Author: Kenneth Reitz <me@kennethreitz.com>
Date:   Thu Jul 31 12:36:35 2014 -0400

    don't print "installing dependencies with pip" twice

commit e72ed7ddbafa0722d2ac6f0cdaf4ac4d0ca0f634
Author: Kenneth Reitz <me@kennethreitz.com>
Date:   Thu Jul 31 12:35:47 2014 -0400

    refactor

commit 64819d9fd7ac042ef56499750b47214eb863f8d2
Author: Kenneth Reitz <me@kennethreitz.com>
Date:   Thu Jul 31 12:24:26 2014 -0400

    pip-grep

commit 5e075dd7dbeda9084ea7ce0151dc17924251c258
Author: Kenneth Reitz <me@kennethreitz.com>
Date:   Thu Jul 31 12:24:13 2014 -0400

    no more travis

commit bc9baba04436dc7b8aefeaee24e820a9720ee9e8
Author: Kenneth Reitz <me@kennethreitz.com>
Date:   Tue Jul 29 13:32:40 2014 -0400

    if not requirement

commit 0918d781e50e6190d1045c19c5c5a37e00d669dc
Author: Kenneth Reitz <me@kennethreitz.com>
Date:   Tue Jul 29 13:28:28 2014 -0400

    only use requirments that are truthy

commit 4f0f750407eae6bc7faea50c7b607a10a11ad1c5
Author: Kenneth Reitz <me@kennethreitz.com>
Date:   Tue Jul 29 13:25:15 2014 -0400

    new version of pip-diff

commit d13b397a21733d0a0a5e31c4995dc6a3b65cd977
Author: Kenneth Reitz <me@kennethreitz.com>
Date:   Tue Jul 29 13:12:23 2014 -0400

    ignore errors for pip-diff

commit bb09fb0d5051f69a35e086030b11fd39d1539f84
Author: Kenneth Reitz <me@kennethreitz.com>
Date:   Tue Jul 29 12:55:55 2014 -0400

    uninstall the packages

commit 29b2e1f2dbe7761e3f7f55827d0adcd0025aeb4b
Author: Kenneth Reitz <me@kennethreitz.com>
Date:   Tue Jul 29 12:47:21 2014 -0400

    maybe this one should be relative

commit 2448558624fdce634350e904eaad5fd3aa725c22
Author: Kenneth Reitz <me@kennethreitz.com>
Date:   Tue Jul 29 12:44:04 2014 -0400

    docopt

commit 47010d297330869516f6add46ace97a41cdccb8d
Author: Kenneth Reitz <me@kennethreitz.com>
Date:   Tue Jul 29 12:40:36 2014 -0400

    first pass at pip-pop integration

commit 59ac5912e1322be771d1969687891ebeb9f4b5f0
Author: Kenneth Reitz <me@kennethreitz.com>
Date:   Tue Jul 22 14:42:08 2014 -0400

    copy requirements
parent 2032942b
Expand all Hide whitespace changes
Inline Side-by-side
Showing with 897 additions and 112 deletions
.travis.yml deleted 100644 → 0
View file @ 2032942b
language: python
python:
- 2.7
script: make tests
notifications:
email: false
\ No newline at end of file
bin/compile
View file @ 47c40ce0
...@@ -37,6 +37,9 @@ LOGPLEX_KEY="t.b90d9d29-5388-4908-9737-b4576af1d4ce" ...@@ -37,6 +37,9 @@ LOGPLEX_KEY="t.b90d9d29-5388-4908-9737-b4576af1d4ce"
export BPWATCH_STORE_PATH=$CACHE_DIR/bpwatch.json export BPWATCH_STORE_PATH=$CACHE_DIR/bpwatch.json
BUILDPACK_VERSION=v28 BUILDPACK_VERSION=v28
# Setup pip-pop (pip-diff)
export PATH=$PATH:$ROOT_DIR/vendor/pip-pop
# Support Anvil Build_IDs # Support Anvil Build_IDs
[ ! "$SLUG_ID" ] && SLUG_ID="defaultslug" [ ! "$SLUG_ID" ] && SLUG_ID="defaultslug"
[ ! "$REQUEST_ID" ] && REQUEST_ID=$SLUG_ID [ ! "$REQUEST_ID" ] && REQUEST_ID=$SLUG_ID
...@@ -144,115 +147,24 @@ set -e ...@@ -144,115 +147,24 @@ set -e
mkdir -p $(dirname $PROFILE_PATH) mkdir -p $(dirname $PROFILE_PATH)
set +e
PYTHON_VERSION=$(cat runtime.txt)
# Install Python. # Install Python.
if [ -f .heroku/python-version ]; then source $BIN_DIR/steps/python
if [ ! $(cat .heroku/python-version) = $PYTHON_VERSION ]; then
bpwatch start uninstall_python
puts-step "Found runtime $(cat .heroku/python-version), removing"
rm -fr .heroku/python
bpwatch stop uninstall_python
else
SKIP_INSTALL=1
fi
fi
if [ ! $STACK = $CACHED_PYTHON_STACK ]; then
bpwatch start uninstall_python
puts-step "Stack changed, re-installing runtime"
rm -fr .heroku/python
unset SKIP_INSTALL
bpwatch stop uninstall_python
fi
if [ ! "$SKIP_INSTALL" ]; then
bpwatch start install_python
puts-step "Installing runtime ($PYTHON_VERSION)"
# Prepare destination directory.
mkdir -p .heroku/python
curl http://lang-python.s3.amazonaws.com/$STACK/runtimes/$PYTHON_VERSION.tar.gz -s | tar zxv -C .heroku/python &> /dev/null
if [[ $? != 0 ]] ; then
puts-warn "Requested runtime ($PYTHON_VERSION) is not available for this stack ($STACK)."
puts-warn "Aborting. More info: https://devcenter.heroku.com/articles/python-support"
exit 1
fi
bpwatch stop install_python # Uninstall removed dependencies with Pip.
source $BIN_DIR/steps/pip-uninstall
# Record for future reference. # Mercurial support.
echo $PYTHON_VERSION > .heroku/python-version source $BIN_DIR/steps/mercurial
echo $STACK > .heroku/python-stack
FRESH_PYTHON=true
hash -r
fi
# If Pip isn't up to date:
if [ "$FRESH_PYTHON" ] || [[ ! $(pip --version) == *$PIP_VERSION* ]]; then
WORKING_DIR=$(pwd)
bpwatch start prepare_environment
bpwatch start install_setuptools
# Prepare it for the real world
# puts-step "Installing Setuptools ($SETUPTOOLS_VERSION)"
cd $ROOT_DIR/vendor/
tar zxf setuptools-$SETUPTOOLS_VERSION.tar.gz
cd $ROOT_DIR/vendor/setuptools-$SETUPTOOLS_VERSION/
python setup.py install &> /dev/null
cd $WORKING_DIR
bpwatch stop install_setuptoools
bpwatch start install_pip
# puts-step "Installing Pip ($PIP_VERSION)"
cd $ROOT_DIR/vendor/
tar zxf pip-$PIP_VERSION.tar.gz
cd $ROOT_DIR/vendor/pip-$PIP_VERSION/
python setup.py install &> /dev/null
cd $WORKING_DIR
bpwatch stop install_pip
bpwatch stop prepare_environment
fi
set -e
hash -r
# Pylibmc support. # Pylibmc support.
# See [`bin/steps/pylibmc`](pylibmc.html). source $BIN_DIR/steps/pylibmc
bpwatch start pylibmc_install
source $BIN_DIR/steps/pylibmc
bpwatch stop pylibmc_install
# Install Mercurial if it appears to be required.
if (grep -Fiq "hg+" requirements.txt) then
bpwatch start mercurial_install
/app/.heroku/python/bin/pip install mercurial | cleanup | indent
bpwatch stop mercurial_install
fi
# Install dependencies with Pip. # Install dependencies with Pip.
puts-step "Installing dependencies with pip" source $BIN_DIR/steps/pip-install
[ ! "$FRESH_PYTHON" ] && bpwatch start pip_install
[ "$FRESH_PYTHON" ] && bpwatch start pip_install_first
/app/.heroku/python/bin/pip install -r requirements.txt --exists-action=w --src=./.heroku/src --allow-all-external | cleanup | indent
[ ! "$FRESH_PYTHON" ] && bpwatch stop pip_install
[ "$FRESH_PYTHON" ] && bpwatch stop pip_install_first
# Django collectstatic support. # Django collectstatic support.
bpwatch start collectstatic source $BIN_DIR/steps/collectstatic
sub-env $BIN_DIR/steps/collectstatic
bpwatch stop collectstatic
# ### Finalize # ### Finalize
# #
......
bin/steps/collectstatic
View file @ 47c40ce0
...@@ -7,6 +7,8 @@ MANAGE_FILE=${MANAGE_FILE:-fakepath} ...@@ -7,6 +7,8 @@ MANAGE_FILE=${MANAGE_FILE:-fakepath}
[ -f .heroku/collectstatic_disabled ] && DISABLE_COLLECTSTATIC=1 [ -f .heroku/collectstatic_disabled ] && DISABLE_COLLECTSTATIC=1
bpwatch start collectstatic
if [ ! "$DISABLE_COLLECTSTATIC" ] && [ -f "$MANAGE_FILE" ]; then if [ ! "$DISABLE_COLLECTSTATIC" ] && [ -f "$MANAGE_FILE" ]; then
set +e set +e
...@@ -28,9 +30,7 @@ if [ ! "$DISABLE_COLLECTSTATIC" ] && [ -f "$MANAGE_FILE" ]; then ...@@ -28,9 +30,7 @@ if [ ! "$DISABLE_COLLECTSTATIC" ] && [ -f "$MANAGE_FILE" ]; then
echo " Collectstatic configuration error. To debug, run:" echo " Collectstatic configuration error. To debug, run:"
echo " $ heroku run python $MANAGE_FILE collectstatic --noinput" echo " $ heroku run python $MANAGE_FILE collectstatic --noinput"
fi fi
echo echo
fi fi
bpwatch stop collectstatic
\ No newline at end of file
bin/steps/mercurial 0 → 100755
View file @ 47c40ce0
# Install Mercurial if it appears to be required.
if (grep -Fiq "hg+" requirements.txt) then
bpwatch start mercurial_install
/app/.heroku/python/bin/pip install mercurial | cleanup | indent
bpwatch stop mercurial_install
fi
\ No newline at end of file
bin/steps/pip-install 0 → 100755
View file @ 47c40ce0
# Install dependencies with Pip.
puts-step "Installing dependencies with pip"
[ ! "$FRESH_PYTHON" ] && bpwatch start pip_install
[ "$FRESH_PYTHON" ] && bpwatch start pip_install_first
/app/.heroku/python/bin/pip install -r requirements.txt --exists-action=w --src=./.heroku/src --allow-all-external | cleanup | indent
# Smart Requirements handling
cp requirements.txt .heroku/python/requirements-declared.txt
/app/.heroku/python/bin/pip freeze > .heroku/python/requirements-installed.txt
[ ! "$FRESH_PYTHON" ] && bpwatch stop pip_install
[ "$FRESH_PYTHON" ] && bpwatch stop pip_install_first
echo
\ No newline at end of file
bin/steps/pip-uninstall 0 → 100755
View file @ 47c40ce0
set +e
# Install dependencies with Pip.
bpwatch start pip_uninstall
if [[ -f .heroku/python/requirements-declared.txt ]]; then
pip-diff --stale .heroku/python/requirements-declared.txt requirements.txt > .heroku/python/requirements-stale.txt
if [[ -s .heroku/python/requirements-stale.txt ]]; then
puts-step "Uninstalling stale dependencies"
/app/.heroku/python/bin/pip uninstall -r .heroku/python/requirements-stale.txt -y --exists-action=w | cleanup | indent
fi
fi
bpwatch stop pip_uninstall
set -e
\ No newline at end of file
bin/steps/pylibmc
View file @ 47c40ce0
...@@ -15,8 +15,10 @@ VENDORED_MEMCACHED="http://cl.ly/0a191R3K160t1w1P0N25/vendor-libmemcached.tar.gz ...@@ -15,8 +15,10 @@ VENDORED_MEMCACHED="http://cl.ly/0a191R3K160t1w1P0N25/vendor-libmemcached.tar.gz
# Syntax sugar. # Syntax sugar.
source $BIN_DIR/utils source $BIN_DIR/utils
bpwatch start pylibmc_install
# If pylibmc exists within requirements, use vendored libmemcached. # If pylibmc exists within requirements, use vendored libmemcached.
if (grep -Eiq "\s*pylibmc" requirements.txt) then if (pip-grep -s requirements.txt pylibmc) then
echo "-----> Noticed pylibmc. Bootstrapping libmemcached." echo "-----> Noticed pylibmc. Bootstrapping libmemcached."
cd .heroku cd .heroku
...@@ -34,5 +36,4 @@ if (grep -Eiq "\s*pylibmc" requirements.txt) then ...@@ -34,5 +36,4 @@ if (grep -Eiq "\s*pylibmc" requirements.txt) then
cd .. cd ..
fi fi
bpwatch stop pylibmc_install
bin/steps/python 0 → 100755
View file @ 47c40ce0
set +e
PYTHON_VERSION=$(cat runtime.txt)
# Install Python.
if [ -f .heroku/python-version ]; then
if [ ! $(cat .heroku/python-version) = $PYTHON_VERSION ]; then
bpwatch start uninstall_python
puts-step "Found runtime $(cat .heroku/python-version), removing"
rm -fr .heroku/python
bpwatch stop uninstall_python
else
SKIP_INSTALL=1
fi
fi
if [ ! $STACK = $CACHED_PYTHON_STACK ]; then
bpwatch start uninstall_python
puts-step "Stack changed, re-installing runtime"
rm -fr .heroku/python
unset SKIP_INSTALL
bpwatch stop uninstall_python
fi
if [ ! "$SKIP_INSTALL" ]; then
bpwatch start install_python
puts-step "Installing runtime ($PYTHON_VERSION)"
# Prepare destination directory.
mkdir -p .heroku/python
curl http://lang-python.s3.amazonaws.com/$STACK/runtimes/$PYTHON_VERSION.tar.gz -s | tar zxv -C .heroku/python &> /dev/null
if [[ $? != 0 ]] ; then
puts-warn "Requested runtime ($PYTHON_VERSION) is not available for this stack ($STACK)."
puts-warn "Aborting. More info: https://devcenter.heroku.com/articles/python-support"
exit 1
fi
bpwatch stop install_python
# Record for future reference.
echo $PYTHON_VERSION > .heroku/python-version
echo $STACK > .heroku/python-stack
FRESH_PYTHON=true
hash -r
fi
# If Pip isn't up to date:
if [ "$FRESH_PYTHON" ] || [[ ! $(pip --version) == *$PIP_VERSION* ]]; then
WORKING_DIR=$(pwd)
bpwatch start prepare_environment
bpwatch start install_setuptools
# Prepare it for the real world
# puts-step "Installing Setuptools ($SETUPTOOLS_VERSION)"
cd $ROOT_DIR/vendor/
tar zxf setuptools-$SETUPTOOLS_VERSION.tar.gz
cd $ROOT_DIR/vendor/setuptools-$SETUPTOOLS_VERSION/
python setup.py install &> /dev/null
cd $WORKING_DIR
bpwatch stop install_setuptoools
bpwatch start install_pip
# puts-step "Installing Pip ($PIP_VERSION)"
cd $ROOT_DIR/vendor/
tar zxf pip-$PIP_VERSION.tar.gz
cd $ROOT_DIR/vendor/pip-$PIP_VERSION/
python setup.py install &> /dev/null
cd $WORKING_DIR
bpwatch stop install_pip
bpwatch stop prepare_environment
fi
set -e
hash -r
\ No newline at end of file
vendor/pip-pop/docopt.py 0 → 100644
View file @ 47c40ce0
This diff is collapsed.
vendor/pip-pop/pip-diff 0 → 100755
View file @ 47c40ce0
#!/usr/bin/env python
# -*- coding: utf-8 -*-
"""Usage:
pip-diff (--fresh | --stale) <reqfile1> <reqfile2>
pip-diff (-h | --help)
Options:
-h --help Show this screen.
--fresh List newly added packages.
--stale List removed packages.
"""
import os
from docopt import docopt
from pip.req import parse_requirements
class Requirements(object):
def __init__(self, reqfile=None):
super(Requirements, self).__init__()
self.path = reqfile
self.requirements = []
if reqfile:
self.load(reqfile)
def __repr__(self):
return '<Requirements \'{}\'>'.format(self.path)
def load(self, reqfile):
if not os.path.exists(reqfile):
raise ValueError('The given requirements file does not exist.')
for requirement in parse_requirements(reqfile):
if requirement.req:
self.requirements.append(requirement.req)
def diff(self, requirements, ignore_versions=False):
r1 = self
r2 = requirements
results = {'fresh': [], 'stale': []}
# Generate fresh packages.
other_reqs = (
[r.project_name for r in r1.requirements]
if ignore_versions else r1.requirements
)
for req in r2.requirements:
r = req.project_name if ignore_versions else req
if r not in other_reqs:
results['fresh'].append(req)
# Generate stale packages.
other_reqs = (
[r.project_name for r in r2.requirements]
if ignore_versions else r2.requirements
)
for req in r1.requirements:
r = req.project_name if ignore_versions else req
if r not in other_reqs:
results['stale'].append(req)
return results
def diff(r1, r2, include_fresh=False, include_stale=False):
include_versions = True if include_stale else False
try:
r1 = Requirements(r1)
r2 = Requirements(r2)
except ValueError:
print 'There was a problem loading the given requirements files.'
exit(os.EX_NOINPUT)
results = r1.diff(r2, ignore_versions=True)
if include_fresh:
for line in results['fresh']:
print line.project_name if include_versions else line
if include_stale:
for line in results['stale']:
print line.project_name if include_versions else line
def main():
args = docopt(__doc__, version='pip-diff')
kwargs = {
'r1': args['<reqfile1>'],
'r2': args['<reqfile2>'],
'include_fresh': args['--fresh'],
'include_stale': args['--stale']
}
diff(**kwargs)
if __name__ == '__main__':
main()
\ No newline at end of file
vendor/pip-pop/pip-grep 0 → 100755
View file @ 47c40ce0
#!/usr/bin/env python
# -*- coding: utf-8 -*-
"""Usage:
pip-grep [-s] <reqfile> <package>...
Options:
-h --help Show this screen.
"""
import os
from docopt import docopt
from pip.req import parse_requirements
class Requirements(object):
def __init__(self, reqfile=None):
super(Requirements, self).__init__()
self.path = reqfile
self.requirements = []
if reqfile:
self.load(reqfile)
def __repr__(self):
return '<Requirements \'{}\'>'.format(self.path)
def load(self, reqfile):
if not os.path.exists(reqfile):
raise ValueError('The given requirements file does not exist.')
for requirement in parse_requirements(reqfile):
self.requirements.append(requirement)
def grep(reqfile, packages, silent=False):
try:
r = Requirements(reqfile)
except ValueError:
if not silent:
print 'There was a problem loading the given requirement file.'
exit(os.EX_NOINPUT)
for requirement in r.requirements:
if requirement.req:
if requirement.req.project_name in packages:
if not silent:
print 'Package {} found!'.format(requirement.req.project_name)
exit(0)
print 'Not found.'.format(requirement.req.project_name)
exit(1)
def main():
args = docopt(__doc__, version='pip-grep')
kwargs = {'reqfile': args['<reqfile>'], 'packages': args['<package>'], 'silent': args['-s']}
grep(**kwargs)
if __name__ == '__main__':
main()
\ No newline at end of file
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment