Python Bytes

By Michael Kennedy and Brian Okken

Listen to a podcast, please open Podcast Republic app. Available on Google Play Store.

Category: Software How-To

Open in iTunes

Open RSS feed

Open Website


Python Bytes is a weekly podcast hosted by Michael Kennedy and Brian Okken. The show is a short discussion on the headlines and noteworthy news in the Python, developer, and data science space.

Episode Date
#82 Let's make a clear Python 3 statement
<p>DigitalOcean: <a href=""><strong></strong></a></p> <p><em>* GitHub coverage coming at the end! *</em></p> <p><strong>Brian #1:</strong> <a href=""><strong>Building and Documenting Python REST APIs With Flask and Connexion</strong></a></p> <ul> <li><strong>Doug Farrell, <a href="">@writeson</a>, on the RealPython site.</strong></li> <li>Tutorial with example. <ul> <li>REST explanation of what REST is and is not</li> <li><a href="">Swagger</a>, swagger.yml to define API</li> <li>Use <a href="">Connexion</a> to incorporate swagger.yml into Flask app.</li> <li>Nice succinct explanation of swagger and API configuration.</li> <li>Demo of Swagger UI for API documentation.</li> <li>JavaScript included for MVC implementation.</li> </ul></li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>MyPy + PyCharm</strong></a></p> <ul> <li>Written by Ivan Levkivskyi</li> <li>via Guido van Rossum</li> <li>Ricky Teachey asks: “What advantages does using mypy bring to pycharm vs just using pycharm's native type checking- which is already pretty good?”</li> <li>Response: <ul> <li>mypy is a bit more stricter/precise</li> <li>it is more configurable, lots of options regulating type system "rules"</li> <li>it typechecks the whole program, so that you immediately see errors your change causes in _other_ files</li> <li>people run mypy in CI and want to see the result before push</li> </ul></li> </ul> <p><strong>Brian #3: Automatic code/doc conversion</strong></p> <ul> <li><a href="">pyupgrade</a> <ul> <li>“A tool (and pre-commit hook) to automatically upgrade syntax for newer versions of the language.”</li> <li>Can even convert to f-strings with <code>--py36-plus</code> option.</li> </ul></li> <li><a href="">docs</a> <ul> <li>“Run <code>black</code> on python code blocks in documentation files.”</li> <li>blacken-docs provides a single executable (blacken-docs) which will modify .rst / .md files in place.</li> </ul></li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>python3statement</strong></a></p> <ul> <li>via Bruno Alla</li> <li>Matthias Bussonnier (Talk Python, <a href="">episode 44</a>) <ul> <li><em>“We now have 44 projects that pledged to drop #python2 in less than 30 months. Some already did ! To see which one, and how to migrate with as few disruption as possible for both Python 2 and 3 users head to ”</em></li> </ul></li> <li>Supporting legacy Python: **it is a small but constant friction in the development of a lot of code.</li> <li>We are keen to use Python 3 to its full potential, and we currently accept the cost of writing cross-compatible code to allow a smooth transition, but we don’t intend to maintain this compatibility indefinitely. </li> <li>Nice “Why switch to Python 3?” section and resources</li> <li>Nice list of participating projects <ul> <li>Can we get some that are not data science? :)</li> </ul></li> </ul> <p><strong>Microsoft buys GitHub</strong>:</p> <ul> <li><a href=";unread=1">Everyone complaining about Microsoft buying GitHub needs to offer a better solution</a></li> <li><a href="">Microsoft to acquire GitHub for $7.5 billion</a></li> <li><a href="">Linux Foundation: Microsoft's GitHub buy is a win for open source</a></li> <li>Coverage on Exponent podcast: <a href="">154 — Legacy Leverage</a></li> <li><a href=";utm_medium=comment_list">Nat Friedman, future CEO of GitHub, AMA</a></li> <li>Re gitlab: <ul> <li><a href="">GitLab congratulates GitHub and Microsoft</a></li> <li><a href="">GitLab imports from GitHub going up</a></li> </ul></li> </ul> <p>Our news and extras:</p> <ul> <li>PyLadies Cleveland just launched: <ul> <li>First meeting June 26</li> <li>(FB Profile) <a href=""></a></li> <li>(FB Community Page) <a href=""></a></li> <li>(Twitter) <a href=""></a></li> <li>(Meetup) <a href=""></a></li> <li>(YouTube) <a href=""></a></li> </ul></li> </ul>
Jun 15, 2018
#81 Making your C library callable from Python by wrapping it with Cython
<p>Sponsored by digitalocean: <a href=""></a></p> <p><strong>Brian #1: Learning about Machine Learning</strong></p> <ul> <li><a href=""><strong>hello tensorflow</strong></a> <ul> <li>one pager site with a demo of machine learning in action.</li> <li>“<strong>Machine Learning (ML)</strong> is the dope new thing that everyone's talking about, because it's really good at learning from data so that it can predict similar things in the future.”</li> <li>Includes a graphical demo of ML trying to learn the correct coefficients to a polynomial.</li> </ul></li> <li><a href="">Google Provides Free Machine Learning Course For All</a> <ul> <li><a href="">Machine Learning Crash Course</a> (MLCC) is a free 15 hours course that is divided into 25 lessons. It provides exercises, interactive visualizations, and instructional videos. These can help in learning machine learning concepts.</li> <li>40 exercises, 25 lessons, 15 hours, case studies, interactive visualizations</li> </ul></li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>Making your C library callable from Python by wrapping it with Cython</strong></a></p> <ul> <li>Article by Stav Shamir</li> <li>Cython is known for its ability to increase the performance of Python code. Another useful feature of Cython is making existing C functions callable from within (seemingly) pure Python modules.</li> <li>Need to directly interact from Python with a small C library</li> </ul> <p>Want to wrap this C function?</p> <pre><code>void hello(const char *name) { printf("hello %s\n", name); } </code></pre> <p>Just install Cython and write this:</p> <pre><code>cdef extern from "examples.h": void hello(const char *name) def py_hello(name: bytes) -&gt; None: hello(name) </code></pre> <p>Then create a setup file (details in article), call <code>python build_ext --inplace</code> and you’re good to go.</p> <p><strong>Brian #3:</strong> <a href=""><strong>Taming Irreversibility with Feature Flags (in Python)</strong></a></p> <ul> <li>“Feature Flags are a very simple technique to make features of your application quickly toggleable. The way it works is, everytime we change some behavior in our software, a logical branch is created and this new behavior is only accessible if some specific configuration variable is set or, in certain cases, if the application context respects some rules.”</li> </ul> <pre><code> def my_function(): if is_feature_active('feature_one'): do_something() else: do_something_else() </code></pre> <ul> <li>Benefits <ul> <li>Improving team’s response time to bugs. If a new feature causes a bunch of customer problems, just turn it off.</li> <li>Making possible to sync code more frequently. Merge to master with the feature turned off.</li> <li>Having a more fluid feature launching flow. Turn feature on in test/staging server.</li> <li>Validate your features easily with A/B testing, user groups, etc.</li> </ul></li> <li>Article discusses: <ul> <li>how to implement flags cleanly.</li> <li>measuring success with analytics</li> <li>implementing flags with third party packages and recommends a few.</li> </ul></li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>pretend: a stubbing library</strong></a></p> <ul> <li>Heard about this at the end of the pypi episode of Talk Python and wanted to highlight it more.</li> <li>Pretend is a library to make stubbing with Python easier.</li> <li>Stubbing is a technique for writing tests. A stub is an object that returns pre-canned responses, rather than doing any computation.</li> <li>Stubbing is related to mocking, but traditionally with stubs, you don’t care about behavior, you are just concerned with how your system under test responds to certain input data. <ul> <li>However, pretend does include a call recorder feature.</li> </ul></li> <li>Nice clean api:</li> </ul> <pre><code> &gt;&gt;&gt; from pretend import stub &gt;&gt;&gt; x = stub(country_code=lambda: "US") &gt;&gt;&gt; x.country_code() 'US' </code></pre> <pre><code> &gt;&gt;&gt; from pretend import stub, raiser &gt;&gt;&gt; x = stub(func=raiser(ValueError)) &gt;&gt;&gt; x.func() Traceback (most recent call last): File "&lt;stdin&gt;", line 1, in &lt;module&gt; File "", line 74, in inner raise exc ValueError </code></pre> <p><strong>Brian #5:</strong> <a href=""><strong>The official Flask tutorial</strong></a></p> <ul> <li>Has been updated recently. <ul> <li>simplified, updated, including the source code for the project.</li> <li>tutorial includes section on testing, including testing with pytest and coverage.</li> </ul></li> <li>Flask is part of <a href="">Pallets</a>, which develops and maintains several projects <ul> <li><a href="">Click</a> — A package for creating beautiful command line interfaces in a composable way</li> <li><a href="">Flask</a> — a flexible and popular web development framework</li> <li><a href="">ItsDangerous</a> — cryptographically sign your data and hand it over to someone else</li> <li><a href="">Jinja</a> — a full featured template engine for Python</li> <li><a href="">MarkupSafe</a> — a HTML-Markup safe string for Python</li> <li><a href="">Werkzeug</a> — a WSGI utility library for Python</li> </ul></li> <li>You can now donate to pallets to help with the maintenance costs of these important packages. <ul> <li>There’s a donate button on the pallets site that takes you to a PSF page. Therefore, donations are deductible in the US.</li> </ul></li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>An introduction to Python bytecode</strong></a></p> <ul> <li>Python is compiled</li> <li>Learn what Python bytecode is, how Python uses it to execute your code, and how knowing what it does can help you.</li> <li>Python is often described as an interpreted language—one in which your source code is translated into native CPU instructions as the program runs—but this is only partially correct. Python, like many interpreted languages, actually compiles source code to a set of instructions for a virtual machine, and the Python interpreter is an implementation of that virtual machine. This intermediate format is called "bytecode."</li> <li>These are your .PYC files</li> </ul> <p>Example:</p> <pre><code>def hello() print("Hello, World!") 2 0 LOAD_GLOBAL 0 (print) 2 LOAD_CONST 1 ('Hello, World!') 4 CALL_FUNCTION 1 </code></pre> <ul> <li>CPython uses a stack-based virtual machine. That is, it's oriented entirely around stack data structures (where you can "push" an item onto the "top" of the structure, or "pop" an item off the "top").</li> </ul> <p>View and explore using</p> <pre><code>import dis dis.dis(hello) </code></pre>
Jun 05, 2018
#80 Dan Bader drops by and we found 30 new Python projects
<p>Sponsored by DigitalOcean: <a href=""><strong></strong></a></p> <p><strong>Brian #1:</strong> <a href=""><strong>Packaging Python Projects</strong></a></p> <ul> <li>Tutorial on the PyPA has been updated.</li> <li>Includes <code></code> instead of <code>REAMDE.rst</code></li> <li>Initial example of <code></code> no longer too minimal or too scary.</li> <li>Discussion of using <code>twine</code> to upload to before uploading to non-test pypi</li> <li><a href="">Related project, flit</a> <a href=""></a></li> </ul> <p><strong>Dan #2:</strong> <a href="">gidgethub — An async library for calling GitHub’s API</a></p> <ul> <li>Talk to GitHub API to add/modify issues, pull-requests, comments, …</li> <li>Also helpers to parse GitHub’s webhook events so you can write bots that react to new issues, comments, commits etc.</li> <li>Used it in @Mariatta’s GitHub Bot tutorial:<a href=""></a></li> <li>Cool architecture for a “modern” Python web API library (async, sansio, decorator based event callbacks) <ul> <li>supports different async backends: aiohttp, treq, Tornado <ul> <li>sans-I/O: “protocol implementations written in Python that perform <strong>no</strong> I/O (this means libraries that operate directly on text or bytes)”</li> <li>Why? → “<em>reusability</em>. By implementing network protocols without any I/O and instead operating on bytes or text alone, libraries allow for reuse by other code regardless of their I/O decisions. In other words by leaving I/O out of the picture a network protocol library allows itself to be used by both synchronous and asynchronous I/O code”</li> </ul></li> </ul></li> <li>(Biggest issue in that workshop was getting everyone upgraded to Python 3.6…but more on that later)</li> </ul> <p><strong>Michael #3:</strong> <a href=""><strong>pysystemd</strong></a></p> <ul> <li>Recall I recently build a Python-based systemd service for geo syncing my course materials</li> <li>A thin Cython-based wrapper on top of libsystemd, focused on exposing the dbus API via sd-bus in an automated and easy to consume way.</li> <li>By Alvaro Leiva, a production engineer at Facebook / Instagram</li> <li><a href="">Presented at PyCon 2018</a></li> <li>Systemd: <ul> <li>Manages your services and their lifetimes</li> <li>e.g. I want my web app to start on boot but only after mongodb has started</li> </ul></li> <li><a href="">pysystemd</a> lets you control and query these from a Python API</li> </ul> <p><strong>Brian #4:</strong> <a href=""><strong>PyCharm 2018.2 EAP 1 includes improved pytest support</strong></a></p> <ul> <li>From <a href="">Bruno Oliveira</a> <ul> <li>“Oh my, full support for <a href="">#</a><a href="">pytest</a> fixtures and parameterized tests coming in <a href="">@</a><a href="">pycharm</a> 2018.2.“</li> </ul></li> <li>“PyCharm 2018.2 supports <a href="">using fixtures in Pytest</a>. Using fixtures allows you to separate your setup code from the actual tests, making for more concise, and more readable tests. Additionally, there have been improvements to code navigation and refactoring Pytest tests, and to using parameterized tests.”</li> <li>It’s hard for me to fully express how FREAKING EXCITED I am about this.</li> <li>auto-complete now works with fixtures to test functions</li> <li>goto declaration now works with fixtures to test functions <ul> <li>(not fixtures of fixtures, but they know about that already)</li> </ul></li> <li>re-running a failed parametrization works (yay!)</li> <li>re-running a single parametrization works (yay!)</li> </ul> <p><strong>Dan #5:</strong></p> <ul> <li>Why is installing Python 3.6 so hard? (Recent GitHub Bot workshop experience)</li> <li>Sometimes hard to tell what’s easy/difficult for beginners</li> <li>People hit crazy edge cases: <ul> <li>running Linux Subsystem for Windows (WSL) on Windows host, install Python into wrong environment</li> <li>broken PPAs + bad StackOverflow advice → broken SSL and no pip on Ubuntu (deadsnakes PPA is the way to go)</li> <li>People install multiple Python environments: Anaconda + distribution</li> <li>Hard to find instructions for compiling from source on Linux</li> </ul></li> <li>Shameless plug: <a href=""><strong></strong></a></li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>30 amazing Python projects (2018 edition)</strong></a> <a href=""></a></p> <ul> <li>Mybridge AI evaluates the quality by considering popularity, engagement and recency. To give you an idea about the quality, the average number of Github stars is 3,707.</li> <li><a href=";utm_medium=blog&amp;utm_campaign=read_more"><strong>No 30</strong></a><strong>:</strong> PDFTabExtract: A set of tools for extracting tables from PDF files helping to do data mining on scanned documents. </li> <li><a href=";utm_medium=blog&amp;utm_campaign=read_more"><strong>No 28</strong></a><strong>:</strong> Surprise v1.0: A Python scikit for building and analyzing recommender systems </li> <li><a href=";utm_medium=blog&amp;utm_campaign=read_more"><strong>No 27</strong></a><strong>:</strong> Eel: A little Python library for making simple Electron-like HTML/JS GUI apps </li> <li><a href=";utm_medium=blog&amp;utm_campaign=read_more"><strong>No 25</strong></a><strong>:</strong> Clairvoyant: A Python program that identifies and monitors historical cues for short term stock movement — Have you seen <a href="">The Wall Street Code - VPRO documentary</a>?</li> <li><a href=";utm_medium=blog&amp;utm_campaign=read_more"><strong>No 21</strong></a><strong>:</strong> Fsociety: Hacking Tools Pack. A Penetration Testing Framework.</li> <li><a href=";utm_medium=blog&amp;utm_campaign=read_more"><strong>No 18</strong></a><strong>:</strong> Maya: Datetime for Humans in Python</li> <li><a href=";utm_medium=blog&amp;utm_campaign=read_more"><strong>No 16</strong></a><strong>:</strong> Better-exceptions: Pretty and useful exceptions in Python, automatically </li> <li><a href=";utm_medium=blog&amp;utm_campaign=read_more"><strong>No 13</strong></a><strong>:</strong> Apistar: A fast and expressive API framework. For Python</li> <li><a href=";utm_medium=blog&amp;utm_campaign=read_more"><strong>No 8</strong></a><strong>:</strong> MicroPython: A lean and efficient Python implementation for microcontrollers and constrained systems</li> <li><a href=";utm_medium=blog&amp;utm_campaign=read_more"><strong>No 6</strong></a><strong>:</strong> spaCy (v2.0): Industrial-strength Natural Language Processing (NLP) with Python and Cython</li> <li><a href=";utm_medium=blog&amp;utm_campaign=read_more"><strong>No 2</strong></a><strong>:</strong> Pytorch: Tensors and Dynamic neural networks in Python with strong GPU acceleration</li> <li><a href=";utm_medium=blog&amp;utm_campaign=read_more"><strong>No 1</strong></a><strong>:</strong> Home-assistant (v0.6+): Open-source home automation platform running on Python 3</li> </ul> <p><strong>Our news</strong></p> <ul> <li>Michael: Notable mention Cris’s GDPR writeup: <a href=""></a> </li> <li>Did you know about dropbox smart sync? <a href=""></a></li> </ul>
May 29, 2018
#79 15 Tips to Enhance your Github Flow
<p>Sponsored by DigitalOcean: <a href=""><strong></strong></a></p> <p><strong>Brian #1:</strong> <a href=""><strong>pytest 3.6.0</strong></a></p> <ul> <li>Revamp the internals of the <code>pytest.mark</code> implementation with correct per node handling which fixes a number of long standing bugs caused by the old design. This introduces new <code>Node.iter_markers(name)</code> and <code>Node.get_closest_mark(name)</code> APIs. - Depricating <code>Node.get_marker(name)</code>. - <a href="">reasons for the revamp</a> - <a href="">updating existing code to use the new APIs</a> - Now when <code>@pytest.fixture</code> is applied more than once to the same function a <code>ValueError</code> is raised. This buggy behavior would cause surprising problems and if was working for a test suite it was mostly by accident.</li> <li>Support for Python 3.7’s builtin <code>breakpoint()</code> method, - see <a href="">Using the builtin breakpoint function</a> for details. - Provided by friend of the show Anthony Shaw</li> <li><code>monkeypatch</code> now supports a <code>context()</code> function which acts as a context manager which undoes all patching done within the <code>with</code> block.</li> <li>whitespace only diffs in failed assertions include escaped characters to be easier to read.</li> <li>plus more… see <a href="">changelog</a></li> </ul> <p><strong>Michael #2: Hello</strong> <a href=""><strong>Qt for Python</strong></a></p> <ul> <li>The first Qt for Python technology preview release is almost here, and for this reason we want to give a brief example on how it will open the doors to the Python world.</li> <li>The real question is: how to access the methods of a Qt class? To simplify the process, we kept Qt APIs. (basically change -> to . in the API)</li> <li>Can it be more pythonic? “We want to include more Python flavor features to Qt for Python in the near future, but at the moment we are focusing on the TP.”</li> <li>The wheels situation: we are planning a set of wheels for Linux, macOS and Windows for 64bit and a 32bit version only for windows. </li> <li>AFAIK, this is Pyside2 reborn</li> </ul> <p><strong>Brian #3:</strong> <a href="!msg/mongodb-user/UWIPZEAKYiw/C5twgBwrCwAJ"><strong>MongoDB 4.0.0-rc0 available</strong></a></p> <ul> <li>MongoDB 4.0.0-rc0, the first release candidate of MongoDB 4.0, is out and is ready for testing. <ul> <li>Multi-document ACID transactions</li> <li>Non-Blocking Secondary Reads</li> <li>lots of other goodies, see <a href="http://MongoDB 4.0.0-rc0, the first release candidate of MongoDB 4.0, is out and is ready for testing.">announcement</a></li> <li>Did we mention <a href="">Transactions</a>!</li> </ul></li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>Pipenv review, after using it in production</strong></a></p> <ul> <li>Nice summary: “The current state of python’s packaging is awful, I don’t think there’s anyone who could disagree with that. This problem is recognized and there are many attempts at trying to solve the mess. Pipenv was the first and it has gained a lot of traction, however it doesn’t sit well with everyone. And it’s also not suited for every project — like libraries.”</li> <li>The multiple environment problem: The tl;dr is — supporting multiple environments goes against Pipenv’s (therefore also Pipfile’s) philosophy of deterministic reproducible application environments. So if you want to use Pipenvfor a library, you’re out of luck. That means many projects just can not use Pipenv for their dependency managment.</li> <li>The good <ul> <li>Pipfile and Pipfile.lock really are superior to requirements.txt. By a ton.</li> <li>While I disliked it at first, having flake8 and security check builtin in a single tool is great</li> <li>Installing (exclusively) from a private respository works very well. Instead of replacing a dotfile somewhere in the system, specifying the repository in Pipfile is great</li> <li>Creating a new Pipfile is very easy</li> <li>No problems introducing Pipenv to it’s new users</li> <li>No problems installing from a mixture of indexes, git repositores…</li> <li>With --sequential it is actually deterministic, as advertised</li> <li>Virtualenv is much easier to get into and understand</li> <li>Dependencies can be installed into system (e.g. in Docker) — our case.</li> <li>At no point did anyone in the team even mentioned getting rid of Pipenv — which is a lot better than it sounds</li> </ul></li> <li>Related: <ul> <li><a href="">PyCon 2018 talk about the history and future of Python packaging, including pipenv</a>.</li> <li><a href="">Recent changes to the official wording around pipenv</a> (removes the statement that it’s the official way of managing <strong>application</strong> dependencies)</li> </ul></li> </ul> <p><strong>Brian</strong> <strong>#5:</strong> <a href=""><strong>15 Tips to Enhance your Github Flow</strong></a></p> <ul> <li>using github projects to prioritize issues and track progress</li> <li>using tags on issues</li> <li>templates</li> <li>using hub and git-extras on command line</li> <li>commit message standards</li> <li>scoped commits</li> <li>style standards with pre-commit hooks</li> <li>automated tests and checks on pull requests</li> <li>protect master branch</li> <li>requiring code reviews</li> <li>squash pull requests</li> <li>…. more great topics</li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>Pandas goes Python 3 only</strong></a></p> <ul> <li>Via Randy Olseon</li> <li>It's official: Starting January 1, 2019, pandas will drop support for #Python 2. This includes no backports of security or bug fixes. </li> <li>Basically following NumPy’s lead</li> <li>The final release before <strong>December 31, 2018</strong> will be the last release to support Python 2. The released package will continue to be available on PyPI and through conda.</li> <li>Starting <strong>January 1, 2019</strong>, all releases will be Python 3 only.</li> </ul> <p><strong>Our news</strong></p> <ul> <li>It’s GDPR eve, are you ready? </li> <li>Need a GDPR laugh? See <a href=""></a></li> <li>XKCD Python Environments: <a href=""></a></li> </ul>
May 25, 2018
#78 Setting Expectations for Open Source Participation
<p>Sponsored by Datadog: <a href=""></a> Special guest: Kojo Idrissa -- <a href=""></a></p> <p>Brian #1: <a href=""><strong>The Forgotten Optional &lt;code&gt;else&lt;/code&gt; in Python Loops</strong></a></p> <ul> <li>“Both for and while loops in Python also take an optional else suite (like the if statement and the try statement do), which executes if the loop iteration completes normally. In other words, the else suite will be executed if we don’t exit the loop in any way other than its natural way. So, no break statements, no return statement, or no exceptions being raised inside the loop.”</li> <li>Why? So you don’t have to invent a flag to indicate something wasn’t found if you are using the loop to search for something. </li> </ul> <p>Kojo #2: <a href=""></a></p> <ul> <li><a href=""></a> <ul> <li>Find out what your dependencies are!</li> <li>Look into <a href=""></a></li> </ul></li> </ul> <p>Michael #3: <a href="">The other (great) benefit of Python type annotations</a></p> <ul> <li>We've had type annotations for awhile</li> <li>When and why is sometimes unclear <ul> <li>Lack of types an issue sometimes, especially annoying while learning new APIs or diving into a new large codebase, and made me completely reliant on documentation.</li> <li>Optional: <ul> <li>You can’t break the code by adding them</li> <li>They have no effect performance-wise</li> <li>You may add them only where you see fit</li> </ul></li> </ul></li> <li>Straightforward benefits <ul> <li>Employ static code analysis to catch type errors prior to runtime</li> <li>Cleaner code/the code is self-documenting: “don’t use a comment when you can use a function or a variable”, we can now say “don’t use comments to specify a type, when you can use type annotation”</li> </ul></li> <li>The other benefit (it's massive!): Code completion</li> </ul> <p>Brian #4: <a href=";;t=48m55s"><strong>Setting Expectations for Open Source Participation</strong></a></p> <ul> <li>Or <strong>Pay for Open Source with Kindness</strong></li> <li>Brett Cannon’s morning talk this last Sunday at PyCon 2018</li> <li>This talk (or a variation of it and it’s content) is essential material for anyone working with open source.</li> <li>Everything in open source has a cost whether it’s time, effort, or emotional output.</li> <li>Open source should be a series of unsolicited kindnesses.</li> <li>Be open, considerate, and respectful</li> <li>Remember most of this runs on volunteer time and that people have lives.</li> <li>Guidelines for communicating online: <ul> <li>Assume you are asking for a favor.</li> <li>Assume your boss will read what you say.</li> <li>Assume your family will read what you say.</li> </ul></li> </ul> <p>Kojo #5:</p> <ul> <li>Python Community Events <ul> <li>Michael and I (along with Trey Hunner) helped lead a New Attendee Orientation</li> <li>Join your local Python community</li> <li>Be kind to your fellow Pythonistas</li> </ul></li> </ul> <p>Michael #6: <a href="">ngrok</a></p> <ul> <li>ngrok exposes local servers behind NATs and firewalls to the public internet over secure tunnels.</li> <li>Public URLs for testing on mobile devices, testing your chatbot, SSH access to your Raspberry Pi, sharing your local dev work on full stack web apps.</li> <li>Just a commandline away</li> <li>My use case: Course app development</li> <li>Features: <ul> <li>Secure Tunnels</li> <li>Request Inspection</li> <li>Fast (HTTP 2)</li> </ul></li> </ul> <p>Extras and our news:</p> <p>Michael:</p> <ul> <li>Live recording video is out: <a href=""></a></li> <li>Now up to 8 video servers around the world, Japan, Sao Paulo, and Mumbai are the latest. Based on the systemd thing we discussed way back when (<a href="">episode 54</a>)</li> </ul>
May 18, 2018
#77 You Don't Have To Be a Workaholic To Win
<p>Sponsored by Datadog: <a href=""><strong>pythonbytes.</strong></a><a href=""><strong>fm</strong></a><a href=""><strong>/datadog</strong></a></p> <p><strong>Brian #1:</strong> <a href=""><strong>Why Senior Devs Write Dumb Code</strong></a></p> <ul> <li>“<em>Any fool can write code that a computer can understand. Good programmers write code that humans can understand.” - Kent Beck</em></li> <li>Code that is clean, straightforward, obvious, and easy to read actually takes practice to achieve.</li> <li>Follow principles like YAGNI, Singe Responsibility, DRY, etc.</li> <li>Avoid clever one-liners, weird abstractions.</li> <li>Esoteric language features.</li> <li>Code needs to be readable and easily understood while under time and stress pressure.</li> </ul> <p><strong>Michael #2:</strong> <a href="">GeoAlchemy 2</a></p> <ul> <li>GeoAlchemy 2 provides extensions to SQLAlchemy for working with spatial databases.</li> <li>GeoAlchemy 2 focuses on PostGIS. Aims to be simpler than its predecessor, GeoAlchemy.</li> <li>Using it: <ul> <li>Connect (e.g. Postgres)</li> <li>Declare a Mapping</li> </ul></li> </ul> <pre><code> class Lake(Base): __tablename__ = 'lake' id = Column(Integer, primary_key=True) name = Column(String) geom = Column(Geometry('POLYGON')) </code></pre> <ul> <li>Create a table (via the engine)</li> <li>Create an Instance of the Mapped Class</li> <li>Inserts like standard SQLAlchmey</li> <li>Spatial Query</li> </ul> <pre><code> from sqlalchemy import func query = session.query(Lake).filter( func.ST_Contains(Lake.geom, 'POINT(4 1)')) query = session.query(, Lake.geom.ST_Buffer(2).ST_Area().label('bufferarea')) </code></pre> <p><strong>Brian</strong> <strong>#3:</strong> <a href=""><strong>QtPyConvert</strong></a></p> <ul> <li>An automatic Python Qt binding transpiler to the abstraction layer.</li> <li>QtPyConvert supports the following bindings out of the box: <ul> <li><a href="">PyQt4</a></li> <li><a href="">PySide</a></li> <li><a href="">PyQt5</a></li> <li><a href="">PySide2</a></li> </ul></li> <li>Conversions leave code comments in place, with the help of RedBaron</li> <li>Converts to <a href=""></a> <ul> <li>Minimal Python 2 &amp; 3 shim around all Qt bindings - PySide, PySide2, PyQt4 and PyQt5</li> </ul></li> </ul> <p><strong>Michael #4:</strong> <a href="">You Don't Have To Be a Workaholic To Win: 13 Alternative Ways To Stand Out</a></p> <ul> <li>Do we have to kill ourselves to get ahead?</li> <li>Don’t busy-brag</li> <li>Max Q analogy</li> <li>The tips</li> <li>Creativity</li> <li>Stubbornness</li> <li>Curiosity</li> <li>Kindness</li> <li>Planning</li> <li>Improvisation</li> <li>Enthusiasm</li> <li>Communication</li> <li>Presence</li> <li>Collaboration</li> <li>Willingness</li> <li>Patience</li> <li>Institutional Knowledge</li> </ul> <p><strong>Brian</strong> <strong>#5:</strong> <a href=""><strong>RedBaron</strong></a></p> <ul> <li>RedBaron is a python library to make the process of writing code that modify source code as easy and as simple as possible. <ul> <li>writing custom refactoring, generic refactoring, tools, </li> </ul></li> <li>Used by QtPyConvert to achieve the conversion while leaving code comments in place</li> <li>Uses the full syntax tree, FST. Like an AST, but keeps all information, including comments and formatting.</li> <li>possible uses: <ul> <li>rename a variable in a source file... without clashing with things that are not a variable (example: stuff inside a string)</li> <li>inline a function/method</li> <li>extract a function/method from a series of line of code</li> <li>split a class into several classes</li> <li>split a file into several modules</li> <li>convert your whole code base from one ORM to another</li> <li>do custom refactoring operation not implemented by IDE</li> </ul></li> </ul> <p><strong>Michael #6:</strong> <a href="">Project Beeware AppStore</a></p> <ul> <li>Project BeeWare has just released its first iPhone app made in Python using its Briefcase tool.</li> <li>Simple travel app for currency and tip calculations</li> <li>Briefcase: A distutils extension to assist in packaging Python projects as standalone applications. Briefcase is a tool for converting a Python project into a standalone native application. You can package projects for: <ul> <li>Mac</li> <li>Windows</li> <li>Linux</li> <li>iPhone/iPad</li> <li>Android</li> <li>AppleTV</li> <li>tvOS</li> </ul></li> <li>While there are other Python GUI toolkits aiming to enable Python developers to build and deploy iOS apps, like for instance the very nice <a href="">Pythonista app</a>, the <a href="">BeeWare</a> project is a bit different because it aims at cross-platform compatibility and native widgets with a set of different tools, like Briefcase and Toga.</li> </ul> <p>Extras: </p> <ul> <li>Michael: Extra ssh breach Did you see that? <a href=""></a></li> <li>PyCon videos already up at <a href=""></a></li> </ul>
May 12, 2018
#76 Goodbye zero-versioning
<p>Sponsored by Datadog: <a href=""><strong>pythonbytes.</strong></a><a href=""><strong>fm</strong></a><a href=""><strong>/datadog</strong></a></p> <p><strong>Brian #1:</strong> <a href=""><strong>Unlearning toxic behaviors in a code review culture</strong></a></p> <ul> <li>unhelpful behaviors: <ul> <li>passing off opinion as fact</li> <li>overwhelming with an avalanche of comments</li> <li>asking people to fix problems they didn’t cause “while they’re at it”.</li> <li>asking judgmental questions</li> <li>being sarcastic</li> <li>using emojis</li> <li>not replying to comments</li> <li>ignoring (not calling out) toxic behavior from high performers</li> </ul></li> <li>helpful: <ul> <li>use questions or recommendations to drive dialog</li> <li>collaborate, don’t back-seat drive</li> <li>respond to every comment</li> <li>know when to take a discussion offline</li> <li>use opportunities to teach, and don’t show off</li> <li>don’t show surprise of lack of knowledge by others</li> <li>automate what can be</li> <li>refuse to normalize toxic behavior</li> <li>managers: hire carefully, listen to your team, and enforce</li> <li>set the standard as your team is small and growing</li> <li>understand you might be part of the problem</li> </ul></li> </ul> <p><strong>Michael #2:</strong> <a href="">Flask 1.0 Released</a></p> <ul> <li>Dropped support for Python 2.6 and 3.3.</li> <li>The CLI is more flexible. <code>FLASK_APP</code> can point to an app factory, optionally with arguments. It understands import names in more cases where filenames were previously used. It automatically detects common filenames, app names, and factory names. <code>FLASK_ENV</code> describes the environment the app is running in, like <code>development</code>, and replaces <code>FLASK_DEBUG</code> in most cases. <a href="">See the docs to learn more.</a></li> <li>If python-dotenv is installed, the <code>flask</code> CLI will load environment variables from <code>.flaskenv</code> and <code>.env</code> files rather than having to export them in each new terminal.</li> <li>The development server is multi-threaded by default to handle concurrent requests during development.</li> <li>flask.ext, which was previously deprecated, is completely removed. Import extensions by their actual package names.</li> <li>Accessing missing keys from <code>request.form</code> shows a more helpful error message in debug mode, addressing a very common source of confusion for developers.</li> <li>Error handlers are looked up by code then exception class, on the blueprint then application. This gives more predictable control over handlers, including being able to handle <code>HTTPException</code>.</li> <li>The behavior of <code>app.logger</code> has been greatly simplified and should be much easier to customize. The logger is always named <code></code>, it only adds a handler if none are registered, and it never removes existing handlers. <a href="">See the docs to learn more.</a></li> <li>The <code>test_client</code> gained a <code>json</code> argument for posting JSON data, and the <code>Response</code> object gained a <code>get_json</code> method to decode the data as JSON in tests.</li> <li>A new <code>test_cli_runner</code> is added for testing an app's CLI commands.</li> <li>Many documentation sections have been rewritten to improve clarity and relevance. This is an ongoing effort.</li> <li>The <a href="">tutorial</a> and corresponding <a href="">example</a> have been rewritten. They use a structured layout and go into more detail about each aspect in order to help new users avoid common issues and become comfortable with Flask. <ul> <li>There are many more changes throughout the framework. <a href="">Read the full</a> </li> </ul></li> </ul> <p><strong>Brian</strong> <strong>#3:</strong> <strong>So, I still don’t quite get pipenv, ….</strong></p> <ul> <li>Best discussion of why pipenv is useful for applications I’ve come across so far is <a href="">Pipenv: A Guide to the New Python Packaging Tool</a></li> <li>Starts with a discussion of situations where <code>pip</code>, <code>pip freeze</code>, and <code>requirements.txt</code> fall apart. <ul> <li><code>requirements.txt</code> often just have an applications direct dependencies, not sub-dependencies.</li> <li><code>pip freeze &gt; requirements.txt</code> will pin your versions to specific versions, but then you’ve got to keep track of dependencies and sub-dependencies.</li> <li><code>Pipfile</code> intends to replace <code>requirements.txt</code>, with a simple-ish human readable format. Also includes extra things like dev environment support.</li> <li><code>Pipfile.lock</code> intends to replace pinned <code>requirements.txt</code> files. Also includes hashes to validate versions haven’t been corrupted.</li> <li><code>pipenv</code> also includes cool tools like dependency graphing, checking for updates, etc.</li> </ul></li> <li><code>pipenv</code> should be used for applications, but not packages intended to be included in other applications. But you can use it during package development, just probably not include the Pipfile and Pipfile.lock in the repo or package distribution. - Brian’s comment</li> </ul> <p>Bonus extra: </p> <ul> <li><a href="">cookiecutter-pipenv: Cookiecutter Python Package Template with Pipenv</a> </li> </ul> <p><strong>Michael #4</strong><a href=""><strong>:</strong></a> <a href="">GraalVM: Run Programs Faster Anywhere</a></p> <ul> <li>Why? <ul> <li>Current production virtual machines (VMs) provide high performance execution of programs only for a specific language or a very small set of languages.</li> <li>Compilation, memory management, and tooling are maintained separately for different languages, violating the ‘don’t repeat yourself’ (DRY) principle.</li> <li>high performance VMs are heavyweight processes with high memory footprint and difficult to embed.</li> </ul></li> <li>Oracle Labs started a new research project for exploring a novel architecture for virtual machines. Our vision was to create a single VM that would provide high performance for all programming languages, therefore facilitating communication between programs.</li> <li>Released: <a href="">GraalVM</a>, a universal virtual machine designed for a polyglot world.</li> <li>GraalVM provides high performance for individual languages and interoperability with zero performance overhead for creating polyglot applications.</li> <li>GraalVM 1.0 allows you to run: <ul> <li>JVM-based languages like Java, Scala, Groovy, or Kotlin</li> <li>JavaScript (including Node.js)</li> <li>LLVM bitcode (created from programs written in e.g. C, C++, or Rust)</li> <li>Experimental versions of Ruby, R, and <strong>Python</strong></li> </ul></li> <li>Future: This first release is only the beginning. We are working on improving all aspects of GraalVM; in particular the support for Python</li> </ul> <p><strong>Brian</strong> <strong>#5:</strong> <a href=""><strong>Testing a Flask Application using pytest</strong></a></p> <ul> <li>Small demo project, and article, that teaches the use of pytest in Flask.</li> <li>unit testing and functional testing</li> <li>Article covers testing models, with an example of a new user.</li> <li>project also has examples of using a test client to check the login page, password authentication, and a lot more.</li> <li>Very cool project.</li> </ul> <p><strong>Bonus:</strong> <a href=""><strong>A cool new pytest plugin: pytest-caprng</strong></a></p> <ul> <li>Tests that use <code>random</code> or <code>np.random</code> may fail, but when you re-run them, they don’t fail, which makes them hard to debug.</li> <li>This plugin adds pytest flags <code>--caprng-global-stdlib</code> and <code>--caprng-global-np</code>, which saves the random state before each test so that if you re-run the test, the random-ness is not so random, and you can reproduce your failure.</li> <li>Also, thanks John for reminding me what “stochastic” means.</li> </ul> <p><strong>Michael #6:</strong> <a href="">How to have a great first PyCon</a></p> <ul> <li>Spending your time: which talks should I go to? The talks at PyCon are typically uploaded to YouTube within 24 hours after the talk ends. I am suggesting that you don’t need to worry about attending every talk.</li> <li>Open spaces: attend them and consider hosting your own! </li> <li>There are a few reasons I often pick open spaces over talks: <ul> <li>Often the open spaces are more niche and topical than the talks: there are some subjects that exist in open spaces every year but which I’ve never seen a talk on</li> <li>Open spaces are all about interaction and discussion whereas talks are a monologue that often evolves into subsequent dialogues</li> <li>Open spaces aren’t recorded whereas the talks are, meaning you can’t really catch up on them later</li> </ul></li> <li>Tips for starting conversation, breakfast and lunch time…</li> <li>The hallway track 👣 <ul> <li>Something you might consider doing while at PyCon is taking breaks in the hallway. </li> <li>In addition to joining or starting a table in the hallway, consider identifying groups that have <a href="">a PacMan opening</a> to join and make sure the groups you’re in are PacMan-friendly.</li> </ul></li> <li>Interacting online during PyCon 🐦 <ul> <li>I recommend getting a Twitter account to make it easier to passively keep up with folks from PyCon after the conference ends.</li> <li>Sometimes people on Twitter will ask if anyone would like to join them for dinner and you might decide to reply and say you’d like to join.</li> </ul></li> <li>Networking isn’t a dirty word: it means making friends 👥</li> <li>I hear two opposing concerns sometimes expressed about PyCon: <ul> <li>Isn’t everyone here to get a job or hire people?</li> <li>Is it acceptable to go to PyCon looking for a job?</li> <li>PyCon is a networking event. That doesn’t necessarily mean everyone is there to get a job, but it also definitely doesn’t mean it’s unacceptable to job-seek at Python.</li> </ul></li> <li>Other topics include <ul> <li>Volunteering</li> <li>Evening events: dinners and board games</li> <li>Give a lightning talk ⚡</li> <li>Take care of yourself</li> </ul></li> <li>Final tip from commentor: If you are on windows, it's helpful to install a virtual image of a linux like the current ubuntu on your laptop, because you could run into situations where you want to follow a talk / training which doesn't work on windows and then you're missing a great opportunity to learn.</li> </ul> <p><strong>Our news</strong></p> <ul> <li>Come see us at PyCon!!! We’ll have stickers! </li> <li>Brian’s talk is Friday at 5 something. </li> <li>We are doing a live Python Bytes open session, join “friends of the show” to get notified</li> <li>I’ll be at Microsoft BUILD too</li> <li>PyGotham 2018 Call for Proposals</li> <li><a href="http://PyCon.DE">http://PyCon.DE</a> (24-26 October 2018 in Karlsruhe, Germany) starting our CfP tomorrow until May 20. <a href=""></a></li> </ul>
May 04, 2018
#75 officially launches
<p>Sponsored by Datadog: <a href=""><strong>pythonbytes.</strong></a><a href=""><strong>fm</strong></a><a href=""><strong>/datadog</strong></a></p> <p><strong>Brian #1:</strong> <a href=""><strong>numba</strong></a></p> <ul> <li>From the numba readme: <ul> <li>“The easiest way to install numba and get updates is by using the Anaconda Distribution: <a href=""></a>”</li> </ul></li> <li><a href=";utm_medium=email&amp;utm_campaign=weekly#The-need-for-speed-without-bothering-too-much:-An-introduction-to-numba">The need for speed without bothering too much: An introduction to numba</a></li> <li>Can get huge speed up for some computation heavy loops or algorithms.</li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>pip 10 is out</strong></a><strong>!</strong></p> <ul> <li>Time for: <code>python -m pip install --upgrade pip</code></li> <li>Features: <ul> <li>Python 2.6 is no longer supported - if you need pip on Python 2.6, you should stay on pip 9, which is the last version to support Python 2.6.</li> <li>Support for PEP 518, which allows projects to specify what packages they require in order to build from source. (PEP 518 support is currently limited, with full support coming in future versions - see the documentation for details).</li> <li>Significant improvements in Unicode handling for non-ASCII locales on Windows.</li> <li>A new "pip config" command.</li> <li>The default upgrade strategy has become "only-if-needed"</li> <li>Many bug fixes and minor improvements.</li> </ul></li> </ul> <p><strong>Brian</strong> <strong>#3:</strong> <a href=""><strong>Keyword (Named) Arguments in Python: How to Use Them</strong></a></p> <ul> <li>Using keyword arguments is often seen when there are many arguments to a function that have useful defaults, and you only want to override the default with some of the arguments.</li> <li>Example:</li> </ul> <div class="codehilite"><pre><span></span><code> <span class="o">&gt;&gt;&gt;</span> <span class="k">print</span><span class="p">(</span><span class="s1">&#39;comma&#39;</span><span class="p">,</span> <span class="s1">&#39;separated&#39;</span><span class="p">,</span> <span class="s1">&#39;words&#39;</span><span class="p">,</span> <span class="n">sep</span><span class="o">=</span><span class="s1">&#39;, &#39;</span><span class="p">)</span> <span class="n">comma</span><span class="p">,</span> <span class="n">separated</span><span class="p">,</span> <span class="n">words</span> </code></pre></div> <ul> <li>You can take positional arguments and require some to be named with various uses of <code>*</code></li> </ul> <pre><code> def foo(*, bar, baz): print(f'{bar} {baz}') </code></pre> <ul> <li>Lots of other useful tricks in this article.</li> </ul> <p><strong>Michael #4</strong><a href=""><strong>:</strong></a> <a href=""><strong> officially launches</strong></a></p> <ul> <li>Legacy PyPI shutting down April 30</li> <li>Listen to talk python 159</li> <li>Starting April 16, the canonical Python Package Index is at <a href=""></a> and uses the new Warehouse codebase.</li> <li>Launched the new PyPI, redirecting browser traffic and API calls (including "pip install") from to the new site. The old codebase is still available at <a href=""></a> for now.</li> <li>Monday April 30 (2018-04-30): We plan to shut down legacy PyPI . The address will continue to redirect to Warehouse.</li> <li>If your site/service links to or uses <a href=""></a>, you should start using instead: <a href=""></a> </li> </ul> <p><strong>Brian</strong> <strong>#5:</strong> <a href=""><strong>Python Modules and Packages – An Introduction</strong></a></p> <ul> <li>In Python, it is, and understanding modules and packages is key to getting a good footing when learning Python. It’s also an area that trips up people when they start trying to create reusable code.</li> <li>How to create a Python <strong>module</strong></li> <li>Locations where the Python interpreter searches for a module</li> <li>How to obtain access to the objects defined in a module with the <code>import</code> statement</li> <li>How to create a module that is executable as a standalone script</li> <li>How to organize modules into <strong>packages</strong> and <strong>subpackages</strong></li> <li>How to control package initialization</li> </ul> <p><strong>Michael #6: Pandas only like modern Python</strong></p> <ul> <li>From December 31st, 2018, Pandas will drop support for Python 2.7. This includes no backports of security or bug fixes (unless someone volunteers to do those)</li> <li>The final release before December 31, 2018 will be the last release to support Python 2. The released package will continue to be available on PyPI and through conda.</li> <li>Starting January 1, 2019, all releases will be Python 3 only.</li> <li>The full <a href="">reddit discussion</a> is interesting.</li> </ul> <p><strong>Our news</strong></p> <ul> <li>Just launched: Python 3, an illustrated tour! <a href=""><strong></strong></a></li> </ul>
Apr 28, 2018
#74 Contributing to Open Source effectively
<p>Sponsored by Datadog: <a href=""><strong>pythonbytes.</strong></a><a href=""><strong>fm</strong></a><a href=""><strong>/datadog</strong></a></p> <p><strong>Special guest: Matt Harrison -</strong> <a href=""><strong>__mharrison__</strong></a></p> <p><strong>Brian #1: Contributing to Open Source effectively</strong> The mechanics and conventions on how to contribute to open source projects can be confusing. After seeing a <a href="">very well documented pull request</a> that started with <code>[WIP]</code> in the subject line when it was first submitted, I tried to find out more about the conventions and mechanics of it all. I’m still learning, but here are a couple of resources:</p> <ul> <li><a href="">How to write the perfect pull request</a> is more of a mindset of how to initiate and receive PRs <ul> <li>Approach to writing a Pull Request, including that [WIP] trick.</li> <li>Offering feedback</li> <li>Responding to feedback</li> </ul></li> <li><a href="">Forge Your Future with Open Source</a>, <a href=""><strong>@</strong></a><a href="">vmbrasseur</a> book on contributing to open source, includes: <ul> <li>Make a Contribution, which includes PRs</li> <li>Make a difference without making a pull request, which is suggests many ways to contribute to a project without contributing code, like reviewing others contributions, testing, triaging bugs, …</li> <li>Interacting with the community.</li> </ul></li> </ul> <p><strong>Matt #2:</strong> <strong>Jupyter, Mathematica, and the Future of the Research Paper</strong></p> <ul> <li>Paul Romer, economy professor at NYU</li> <li>As a longtime Linux user there was constantly the question of the “year of the Linux Desktop”. Maybe this is the year of the “Jupyter desktop” (also beta version of JupyterLab). Not just a tool for innovators or early adopters</li> <li>Refers to Article in Atlantic contrasting Mathematica and Jupyter: <em>open-source developers have flocked to Python because it happens to be the de facto standard for scientific computing. Programming-language communities, like any social network, thrive—or die—on the strength of these feedback loops</em>. <a href=""> </a></li> </ul> <p>Quotes: </p> <p><em>Jupyter is a new open-source alternative [to Mathmatica] that is well on the way to becoming a standard for exchanging research results.</em></p> <p><em>Python libraries let me replicate everything I wanted to do with Mathematica: Matplotlib for graphics, SymPy for symbolic math, NumPy and SciPy for numerical calculations, Pandas for data, and NLTK for natural language processing. Jupyter makes it easy to use Latex to display typeset math. With Matplotlib, Latex works even in the label text for graphs. (I have not yet tried the major update, JupyterLab, which is still in beta testing.)</em> <em>I’m more productive. I’m having fun.</em> <a href=""></a></p> <p><strong>Michael</strong> <strong>#3:</strong> <a href=""><strong>Python Developers Survey 2017 Results</strong></a></p> <ul> <li>At the very end of 2017, JetBrains &amp; The PSF teamed up to build a solid picture of the modern Python developer</li> <li>Here are some take-aways <ul> <li>Almost 4 out of 5 Python developers use it as their main language, while for 21% it’s only a secondary language.</li> <li>Data analysis is as popular as web development with Python: Web development is the only category with a large gap (54% vs 33%) separating those using Python as their main language vs as a supplementary language. For other types of development, the differences are far less significant.</li> <li>At 28% to 27% application, There are as many Python web developers as Python data scientists</li> <li>Python 3 vs Python 2: 75% to 25% and accelerating</li> <li>Top Cloud Platform(s) <ul> <li>67%: AWS</li> <li>29%: Google App Engine</li> <li>26%: Heroku</li> <li>23%: DigitalOcean</li> <li>16%: Microsoft Azure</li> </ul></li> <li>Team Size <ul> <li>74%: 2-7 people</li> <li>16%: 8-12 people</li> <li>5%: 13-20 people</li> <li>2%: 21-40 people</li> <li>2%: &gt; 40 people</li> </ul></li> <li>Operating Systems <ul> <li>49%: Windows</li> <li>19%: Linux</li> <li>15%: MacOS</li> </ul></li> </ul></li> </ul> <p><strong>Brian #4</strong><a href=""><strong>:</strong></a> <a href=""><strong><em>*</a><a href=""></strong>EdgeDB: A New Beginning</em>*</a> This is “news you can’t use” so far, because the product isn’t here yet. So why am I excited and interested in this:</p> <ul> <li>It’s from Elvis <a href=""><strong>@</strong></a><a href="">elprans</a> and Yury <a href=""><strong>@</strong></a><a href="">1st1</a>, who have brought us asyncio and uvloop</li> <li>It’s not just a relational DB, it’s a DB based on PostgreSQL but with an entire new way to specify schema and interact with it.</li> <li>Goal is to be fast, user friendly, and remove the need for ORMs</li> </ul> <p><strong>Matt</strong> <strong>#5:</strong> <a href=""><strong>Yellowbrick library</strong></a> </p> <ul> <li>Visualization is important, I’ve found bugs by plotting before. Also important in evaluation of machine learning projects</li> <li>This is a project that has been around for about two years. I’ve recently adopted it in place of some home grown libraries for some consulting projects and in my corporate training</li> <li>Yellowbrick offers visualization for: <ul> <li>Features</li> <li>Classification</li> <li>Regression</li> <li>Clustering</li> <li>Text</li> </ul></li> <li>Like sk-learn, uses a similar api (.fit, .transform, .poof (plot))</li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>Depression AI</strong></a></p> <ul> <li>Alexa skill for people suffering with depression. </li> <li>Alexa <a href=";ie=UTF8&amp;qid=1522775645&amp;sr=1-1&amp;keywords=depression+ai">store listing</a></li> <li>Based on <a href="">Flask-Ask</a> <ul> <li>Discussed on <a href="">Talk Python 146</a></li> </ul></li> <li>Valley Hackathon 2018 winner</li> <li>71% of people who make their bed in the morning report feeling happy. This was the inspiration behind DepressionAI. </li> <li>The aim behind this skill is to encourage people to perform daily activities that become very difficult when one is depressed.</li> <li>The skill detects positive and negative moods. </li> <li>If the user is having a bad day, it asks them a series of questions about what they have done that day (e.g. "Have you gotten out of bed?") and if they haven't, it encourages them to do so.</li> <li>Features <ul> <li>Mood evaluation by a highly empathetic Alexa bot</li> <li>Suicidal intention detection and prevention attempt</li> <li>Location-based therapy reccomendations</li> <li>Suggestions for small activites to improve the user's mood</li> <li>Displays informative cards in the Alexa app</li> </ul></li> <li>Sample Phrases <ul> <li>“Alexa, check on me."</li> <li>"I feel down."</li> <li>"I haven't got out of bed today."</li> <li>"Help me feel better."</li> <li>"Help me find a therapist"</li> </ul></li> </ul>
Apr 19, 2018
#73 This podcast comes in any color you want, as long as it's black
<p>Sponsored by Datadog: <a href=""><strong>pythonbytes.</strong></a><a href=""><strong>fm</strong></a><a href=""><strong>/datadog</strong></a></p> <p><strong>Brian #1:</strong> <a href=""><strong>Set Theory and Python</strong></a></p> <ul> <li>“Let’s talk about sets, baby …” is what I have in my head while reading this.</li> <li>Great overview of set theory and how to use the set data type in Python.</li> <li>Covered: <ul> <li>Creating sets</li> <li>Checking for containment (in, not in)</li> <li>union : set of things in either set or in both</li> <li>intersection: set of things in 2 sets</li> <li>difference: set of things in one set but not the other</li> <li>symmetric difference: set of things in either set but not in both</li> </ul></li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>Trio: async programming for humans and snake people</strong></a></p> <ul> <li>The Trio project’s goal is to produce a production-quality, permissively licensed, async/await-native I/O library for Python. Like all async libraries, its main purpose is to help you write programs that do multiple things at the same time with parallelized I/O. </li> <li>Compared to other libraries, Trio attempts to distinguish itself with an obsessive focus on usability and correctness. </li> <li>Concurrency is complicated; we try to make it easy to get things right.</li> <li>Trio was built from the ground up to take advantage of the latest Python features</li> <li>Inspiration from <a href="">many sources</a>, in particular Dave Beazley’s <a href="">Curio</a></li> <li>Resulting design is radically simpler than older competitors like asyncio and Twisted, yet just as capable.</li> <li>We <em>do</em> encourage you do use it, but you should <a href="">read and subscribe to issue #1</a> to get warning and a chance to give feedback about any compatibility-breaking changes.</li> <li>Excellent scalability: trio can run 10,000+ tasks simultaneously without breaking a sweat, so long as their total CPU demands don’t exceed what a single core can provide.</li> <li>Supports Python 3.5+ and PyPy</li> <li>Uses</li> </ul> <pre><code>, 3) trio.sleep(1.5) # Sleep, non-blocking async with trio.open_nursery() as nursery: print("parent: spawning child...") nursery.start_soon(child_func1) print("parent: spawning child...") nursery.start_soon(child_func2) print("parent: waiting for children to finish...") # -- we exit the nursery block here -- print("parent: child_func1 and child_func2 done!") </code></pre> <ul> <li>trio provides a <a href="">rich set of tools for inspecting and debugging your programs</a>.</li> <li>Consider <a href="">trio-asyncio</a> for compatibility</li> </ul> <p><strong>Brian #3:</strong> <a href=""><strong>black: The uncompromising Python code formatter</strong></a></p> <ul> <li><p>An amusing take on code formatting. From the readme:</p> <ul> <li>“<em>Black</em> is the uncompromising Python code formatter. By using it, you agree to cease control over minutiae of hand-formatting. In return, <em>Black</em> gives you speed, determinism, and freedom from <code>pycodestyle</code> nagging about formatting. You will save time and mental energy for more important matters.”</li> <li>“Blackened code looks the same regardless of the project you're reading. Formatting becomes transparent after a while and you can focus on the content instead.”</li> <li>“<em>Black</em> makes code review faster by producing the smallest diffs possible.”</li> </ul></li> <li><p>Datadog is a monitoring solution that provides deep visibility and tracks down issues quickly with distributed tracing for your Python apps.</p></li> <li>Within minutes, you'll be able to investigate bottlenecks in your code by exploring interactive flame graphs and rich dashboards.</li> <li>Visualize your Python performance today, get started with a free trial with Datadog and they'll send you a free T-shirt.</li> </ul> <p>See for yourself, visit </p> <p><strong>Michael #4:</strong> <a href=""><strong>gain: Web crawling framework based on asyncio</strong></a></p> <ul> <li>Web crawling framework for everyone. Written with asyncio, uvloop and aiohttp.</li> <li>Simple and mostly automated <ul> <li>Define class mapped to CSS selectors and data to save</li> <li>Concurrently level</li> <li>Start URL</li> <li>Page templates to match URLs</li> <li>Run</li> </ul></li> </ul> <p><strong>Brian #5:</strong> <a href=""><strong>Generic Function in Python with Singledispatch</strong></a></p> <ul> <li>“Imagine, you can write different implementations of a function of the same name in the same scope, depending on the types of arguments. Wouldn’t it be great? Of course, it would be. There is a term for this. It is called “Generic Function”. Python recently added support for generic function in Python 3.4 (<a href="">PEP 443</a>). They did this to the <code>functools</code> module by adding <code>@singledispatch</code> decorator.”</li> <li>For people less familiar with “generic functions”. I think of this as providing similar functionality as C++’s function overloading.</li> <li>Allows you do things like this (full code example is in the article):</li> </ul> <pre><code> from functools import singledispatch @singledispatch def fprint(data): "code for default functionality" @fprint.register(list) @fprint.register(set) @fprint.register(tuple) def _(data): "code for list, set, tuple" @fprint.register(dict) def _(data): "code for dict" </code></pre> <p>More complete code example: </p> <pre><code> from functools import singledispatch @singledispatch def fprint(data): print(f'({type(data).__name__}) {data}') @fprint.register(list) @fprint.register(set) @fprint.register(tuple) def _(data): formatted_header = f'{type(data).__name__} -&gt; index : value' print(formatted_header) print('-' * len(formatted_header)) for index, value in enumerate(data): print(f'{index} : ({type(value).__name__}) {value}') @fprint.register(dict) def _(data): formatted_header = f'{type(data).__name__} -&gt; key : value' print(formatted_header) print('-' * len(formatted_header)) for key, value in data.items(): print(f'({type(key).__name__}) {key}: ({type(value).__name__}) {value}') # &gt;&gt;&gt; fprint('hello') # (str) hello # &gt;&gt;&gt; fprint(21) # (int) 21 #... # &gt;&gt;&gt; fprint({'name': 'John Doe', 'age': 32, 'location': 'New York'}) # dict -&gt; key : value # ------------------- # (str) name: (str) John Doe # (str) age: (int) 32 # (str) location: (str) New York </code></pre> <p><strong>Michael #6:</strong> <a href=""><strong>Unsync: Unsynchronizing async/await in Python 3.6</strong></a></p> <ul> <li>A rant about async/await in Python (by Alex Sherman)</li> <li>What’s wrong? <ul> <li>The two big friction points I’ve had are: <ul> <li>Difficult to “fire and forget” async calls (need to specifically run the event loop)</li> <li>Can’t do blocking calls to asyncio.Future.result() (it throws an exception)</li> </ul></li> <li>We need to acquire an even loop, do some weird call to execute the async function in that event loop, and then synchronously execute the event loop ourselves. </li> </ul></li> <li>What can we do? <ul> <li>C# had this great idea of executing each Task (their version of a Future) first synchronously in the main thread until an await is hit, and then queueing it into an ambient thread pool to continue later possibly in a separate thread. </li> <li>Python did not take this approach and my hunch is that the Python maintainers didn’t want to add an ambient thread pool to their language (which makes sense). </li> <li>Alex, however, is not the Python maintainers and did add an ambient thread (singular). I stuffed all the boiler plate into a decorator and the result looks like this:</li> </ul></li> </ul> <pre><code> @unsync async def unsync_async(): await asyncio.sleep(0.1) return 'I like decorators' print(unsync_async().result()) </code></pre> <ul> <li>using @unsync on a regular function (not an async one) will cause it to be executed in a ThreadPoolExecutor. </li> <li>To support CPU bound workloads, you can use @unsync(cpu_bound=True) to decorate functions which will be executed in a ProcessPoolExecutor</li> </ul>
Apr 12, 2018
#72 New versioning: Episode (with 72 releases)
<p>Sponsored by Datadog: <a href=""><strong></strong></a></p> <p><strong>Brian #1:</strong> <a href=""><strong>ZeroVer: 0-based Versioning</strong></a></p> <ul> <li>“Software's most popular versioning scheme!”</li> <li>“<em>Cutting-edge software versioning for minimalists”</em></li> <li>My favorite April Fools prank this year. <ul> <li>Calls out many popular projects for never reaching 1.0</li> </ul></li> <li>From the about page: <ul> <li>“ZeroVer is the world's most popular software versioning convention, and the only one shown to harness the innovative power of zero. The benefits are innumerable and the effects on the software world are profound.”</li> <li>“Version 0.0.1 of ZeroVer was published by <a href="">Mahmoud Hashemi</a>, with help from Moshe, Mark, Kurt, and other patient collaborators, on 2018-04-01. ZeroVer is satire, <a href="">please do not use it</a>. We sincerely hope no project release schedules were harmed as a result of this humble attempt at programmer humor.”</li> </ul></li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>GitHub Security Alerts Detected over Four Million Vulnerabilities</strong></a></p> <ul> <li>Last year GitHub launched “GitHub security alerts”</li> <li>GitHub’s security alerts notify repository admins when library vulnerabilities from the Common Vulnerabilities and Exposures (CVEs) list are detected in their repositories. </li> <li>Nearly half of all displayed alerts are responded to within a week and the rate of vulnerabilities resolved in the first seven days has been about 30%. </li> <li>When that statistics is restricted to only repositories with recent contributions, i.e., contributions in the last 90 days, things look even brighter, GitHub says, with 98% of such repositories being patched in fewer than seven days. </li> <li>More than four million vulnerabilities in over 500,000 repositories have been reported.</li> <li>Security alerts are only currently supported for repositories written in Ruby or JavaScript, while support for Python is planned for 2018.</li> <li>I also recommend <a href=""></a></li> </ul> <p><strong>Brian #3:</strong> <a href=""><strong>Markdown Descriptions on PyPI</strong></a></p> <ul> <li>Dustin Ingram provides detailed steps on how to get this to work.</li> <li> now supported by <ul> <li>“Only <a href=""></a> will correctly render your new Markdown description.</li> <li>Legacy PyPI (<a href=""></a>) will still render your description as plaintext, but don’t worry, <a href="">it’s going away real soon</a>.</li> </ul></li> <li>And also, <a href="">Github-Flavored Markdown Descriptions are supported</a>. <ul> <li>Another post, this one by Jon Wayne Parrot</li> </ul></li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>Concurrency comparison between NGINX-unit and uWSGI</strong></a></p> <ul> <li>show performance of two web application servers <ul> <li>nginx-unit (a new modern application web server)</li> <li>uWSGI (the best one application server)</li> </ul></li> <li>uWSGI and nginx-unit configured with 4 workers because test system has 4 cores.</li> <li>Effectively an empty “Hello world” Flask app</li> <li>Have a look at the pictures here:</li> <li>Take away: I’m going to start paying attention to NGINX-unit</li> </ul> <p><strong>Brian #5:</strong> <a href=""><strong>Loop better: A deeper look at iteration in Python</strong></a></p> <ul> <li>via Trey Hunner</li> <li>Generators are a great way to loop, but have a few gotchas <ul> <li>Looping twice doesn’t work</li> <li>Containment checks muck up the generator “contents”.</li> <li>Unpacking has odd results.</li> </ul></li> <li>This article describes Python loops in detail and then applies that do describe why the gotchas act like they do.</li> <li>Covered: <ul> <li>iterators, iterables, sequences, generators</li> <li>the iterator protocol</li> </ul></li> <li>Reading this will make you a better programmer, but might hurt your head.</li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>Misconfigured Django Apps Are Exposing Secret API Keys, Database Passwords</strong></a></p> <ul> <li>Security researchers have been stumbling upon misconfigured Django applications that are exposing sensitive information such as API keys, server passwords, or AWS access tokens.</li> <li>He discovered 28,165 Django apps just this week where admins left debug mode enabled.</li> <li>Just by skimming through a few of the servers, the researcher found that the debug mode of many of these apps were exposing extremely sensitive information that would have allowed a malicious actor full access to the app owner's data. </li> <li>This is <strong>not a failure from Django's side</strong>. My recommendation is to disable debugging mode when deploying the application to production.</li> <li>Security researcher Victor Gevers said some of the servers running Django apps have already been compromised.</li> <li>He found at least one compromised server, running the Weevely web shell. Some servers Gevers found leaking sensitive data belonged to various government agencies carrying out critical operations.</li> <li>Gevers said he started notifying servers owners about their leaky Django apps. "At this moment we have reported 1,822 servers," Gevers said. "143 were fixed or taken offline."</li> </ul> <p><strong>Extra:</strong></p> <ul> <li>We covered wagtail on <a href="">episode 70</a>. They are running <a href="">a kickstarter campaign</a> to get some new features out. There’s a video there.</li> </ul>
Apr 05, 2018
#71 We can migrate to Python 3, careful please
<h1>Python Bytes 71</h1> <p>Sponsored by DigitalOcean: <strong><a href=""></a></strong></p> <p>Special guest: Trey Hunner (<a href=""><strong>@treyhunner</strong></a>)</p> <p><strong>Trey #1:</strong> <a href=""><strong>The Conservative Python 3 Porting Guide</strong></a></p> <ul> <li>by <a href="">various Red Hat folks</a> mostly</li> <li>Python 2 is coming to the end of its life on January 1, 2020. Are you ready?</li> <li>This is one of the best guides I’ve found to porting your code from Python 2 to Python 3</li> <li>One of the issues with many of the Python 3 porting guides is that the old ones recommend dropping Python 2 support suddenly, which isn’t recommended anymore.</li> <li>I do wish this guide recommended the future library instead of python-modernize. They’re both great, but modernize is a little less focused on writing things the Python 3 way and a little more focused on just getting your code working in both 2 and 3.</li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>World-Class Software Companies That Use Python</strong></a></p> <ul> <li>by <a href="">Jason Reynolds</a> </li> <li>While it’s easy to see how you can tinker with Python, you might be wondering how this translates to actual business and real world applications.</li> <li><strong>Industrial Light and Magic</strong> <ul> <li>The studio has used Python in multiple other facets of their work. Developers use Python to track and audit pipeline functionality, maintaining a database of every image produced for each film.</li> </ul></li> <li><strong>Google</strong> <ul> <li>In the beginning, the founders of Google made the decision of “Python where we can, C++ where we must.”</li> <li>Currently powers YouTube among other things</li> </ul></li> <li><strong>Facebook</strong> <ul> <li>Ensures that the infrastructure of Facebook is able to scale efficiently</li> </ul></li> <li><strong>Instagram</strong> <ul> <li>the Instagram engineering team boasted that they were <a href="">running the world’s largest deployment of the Django web framework, which is written entirely in Python</a>.</li> <li>Instagram’s engineering team has invested time and resources into keeping their Python deployment viable at the massive scale (<a href="">~800 million monthly active users</a>) they’re operating at.</li> <li><a href="">PyCon 2017 keynote talk</a> by Lisa Guo and Hui Ding</li> </ul></li> <li><strong>Spotify</strong> <ul> <li>This music streaming giant is a <a href="">huge proponent of Python</a>, using the language primarily for data analysis and back end services. </li> <li>On the back end, there are a large number of services that all communicate over 0MQ, or <a href="">ZeroMQ</a>, an open source networking library and framework that is written in Python and C++(among other languages).</li> </ul></li> <li><strong>Quora</strong> <ul> <li>choosing to use Python where they could because of its ease of writing and readability, and implemented C++ for the performance critical sections. </li> <li>They got around Python’s lack of typechecking by writing unit tests that accomplish much the same thing.</li> <li>Another key consideration for using Python was the existence of several good frameworks at the time including Django and Pylons. </li> </ul></li> <li><strong>Netflix</strong> <ul> <li>Lots of infrastructure and ops work done via Python</li> </ul></li> <li><strong>Dropbox</strong> <ul> <li>Dropbox makes heavy use of Python</li> <li>Guido van Rossum works there!</li> <li>Lots of open source projects</li> <li>Client app in Python too</li> </ul></li> <li><strong>Reddit</strong> <ul> <li>This website had 542 million visitors every month across 2017, making it the fourth most visited website in the United States and seventh most visited in the world. </li> <li>In 2015, there were 73.15 million submissions and 82.54 billion pageviews. </li> <li>Behind it all, forming the software backbone, was Python.</li> </ul></li> </ul> <p><strong>Trey #3:</strong> <a href=""><strong>Stop Writing Classes</strong></a></p> <ul> <li>by <a href="">Jack Diederich</a></li> <li>This is one of my favorite PyCon talks to recommend to folks switching to Python from other programming languages. I especially like to recommend this talk to folks moving to Python from Java and C++.</li> <li>This is kind of an old talk. It's from 2012, so it's from the days of Python 2 but everything in it is still <em>very applicable today</em>.</li> <li>One of the great things about this talk is it doesn’t just show times that you should write functions instead of classes, it also shows an example or two of when classes really make sense.</li> <li>The big advice from this talk: if you have a class that only has two methods and one is the initializer, you probably need a function instead.</li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong> is alive</strong></a></p> <ul> <li>For the LONGest time, pypi has been run out of <a href=""></a></li> <li>Now the new version of pypi is out at</li> <li>Rewritten in Pyramid</li> <li>Do you want to contribute? Now the barriers have come down</li> <li><strong><a href="">Tweet with graphs</a></strong></li> </ul> <p><strong>Trey</strong> <strong>#5:</strong> <a href=""><strong>Pragmatic Unicode</strong></a></p> <ul> <li>by <a href="">Ned Batchelder</a></li> <li>Another PyCon 2012 talk that is still relevant today, though it does use quite a bit of Python 2 syntax</li> <li>Ned describes the unicode sandwich in this talk. Talks with good metaphors really help shape your mental model of a topic. This was the talk that helped me really understand the unicode vs bytes issue that Python 3 largely solves for us (or at least forces us to do so upfront).</li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>pygame on pypy usable</strong></a></p> <ul> <li>via René Dudfield</li> <li>0.5x to 30x the speed</li> <li>That is pygame (same one that runs on cpython), works on pypy through its C extension API</li> <li><strong>This is exciting because</strong>: <ul> <li>pure python code being fast on pypy(after warmup), also mixed with the fast bits in C/asm.</li> <li>cpyext is getting faster in pypy. There is already work and discussion towards it being faster than CPython.</li> <li>maintaining one pygame code base is easier than maintaining several (pygame cffi/ctypes/cython, ...).</li> <li>with one code base it should be fast on both pygame, and pypy(in time).</li> </ul></li> <li><strong>Where it can be slower</strong>: if you are going into C code for a lot of small operations. Like when using lots of pygame.Rect in a tight loop. This is because (currently) the cost of going from PyPy code into and out of CPython API code (like pygame) is a bit slow.</li> <li>Ray tracing in PyGame: <ul> <li>On PyPy - 18.6 seconds.</li> <li>On Python 2.7 - 9 minutes, 28.1 seconds (30x slower)</li> </ul></li> </ul> <p><strong>Follow up and other news</strong></p> <p><strong>Michael:</strong> <strong>#100DaysOfCode in Python course</strong>: <strong><a href=""></a></strong></p> <p><strong>Trey:</strong> Python Morsels: <strong><a href=""></a></strong></p>
Mar 28, 2018
#70 Have you seen my log? It's cute!
<p>Sponsored by DigitalOcean: <a href=""><strong></strong></a></p> <p><strong>Brian #1:</strong> <a href=""><strong>Online CookieCutter Generator</strong></a></p> <ul> <li>“Get a ZIP-archive with project by filling out the form.”</li> <li>By <a href="">@kpavlovsky_pro</a> Konstantin Pavlovsky</li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>cutelog – GUI for Python's logging module</strong></a></p> <ul> <li>This is a graphical log viewer for Python's standard logging module. </li> <li>Features <ul> <li>Allows any number of simultaneous connections</li> <li>Fully customizable look of log levels and columns</li> <li>Filtering based on level and name of the logger, as well as filtering by searching</li> <li>Search through all records or only through filtered ones</li> <li>View exception tracebacks or messages in a separate window</li> <li>Dark theme (with its own set of colors for levels)</li> <li>Pop tabs out of the window, merge records of multiple tabs into one</li> </ul></li> <li>Based on PyQt5 speaking of GUIs</li> </ul> <p><strong>Brian #3:</strong> <a href=""><strong>wagtail 2.0</strong></a></p> <ul> <li>“Wagtail is a content management system built on Django. It’s focused on user experience, and offers precise control for designers and developers.”</li> <li><a href="">The Zen of Wagtail</a> - nice philosophy of the project page to let you know if this kind of thing is right for you and your project.</li> <li>In 2.0 <ul> <li>a new text editor</li> <li>Django 2 support </li> <li>better scheduled publishing</li> <li>…</li> </ul></li> <li><a href="">wagtail docs</a></li> <li><a href="">gallery of sites made with wagtail</a></li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>peewee 3.0 is out</strong></a></p> <ul> <li>Peewee is a simple and small ORM. It has few (but expressive) concepts, making it easy to learn and intuitive to use. <ul> <li>A small, expressive ORM</li> <li>Written in python with support for versions 2.7+ and 3.4+ (developed with 3.6)</li> <li>Built-in support for SQLite, MySQL and Postgresql.</li> <li>Numerous extensions available (postgres hstore/json/arrays, sqlite full-text-search, schema migrations, and much more).</li> </ul></li> <li>Although this was pretty much a complete rewrite of the 2.x codebase, I have tried to maintain backwards-compatibility for the public APIs.</li> <li>Exciting because of its async support via <strong>peewee-async</strong> <ul> <li><strong>peewee-async</strong> is a library providing asynchronous interface powered by <a href="">asyncio</a> for <a href="">peewee</a> ORM.</li> </ul></li> </ul> <div class="codehilite"><pre><span></span><code> <span class="n">database</span><span class="o">.</span><span class="n">set_allow_sync</span><span class="p">(</span><span class="bp">False</span><span class="p">)</span> <span class="n">async</span> <span class="k">def</span> <span class="nf">handler</span><span class="p">():</span> <span class="n">await</span> <span class="n">objects</span><span class="o">.</span><span class="n">create</span><span class="p">(</span><span class="n">TestModel</span><span class="p">,</span> <span class="n">text</span><span class="o">=</span><span class="s2">&quot;Not bad. Watch this, I&#39;m async!&quot;</span><span class="p">)</span> <span class="n">all_objects</span> <span class="o">=</span> <span class="n">await</span> <span class="n">objects</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="n">TestModel</span><span class="o">.</span><span class="n">select</span><span class="p">())</span> <span class="k">for</span> <span class="n">obj</span> <span class="ow">in</span> <span class="n">all_objects</span><span class="p">:</span> <span class="k">print</span><span class="p">(</span><span class="n">obj</span><span class="o">.</span><span class="n">text</span><span class="p">)</span> </code></pre></div> <p><strong>Brian #5:</strong> <a href=""><strong>Machine Learning Basics</strong></a></p> <ul> <li>“Plain python implementations of basic machine learning algorithms”</li> <li>From the repo: <ul> <li>A repository of implementations of basic machine learning algorithms in plain Python (Python Version 3.6+). All algorithms are implemented from scratch without using additional machine learning libraries. The intention of these notebooks is to provide a basic understanding of the algorithms and their underlying structure, not to provide the most efficient implementations. <ul> <li>Linear Regression</li> <li>Logistic Regression</li> <li>Perceptron</li> <li>k-nearest-neighbor</li> <li>k-Means clustering</li> <li>Simple neural network with one hidden layer</li> <li>Multinomial Logistic Regression</li> </ul></li> </ul></li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>Cerberus</strong></a></p> <ul> <li>Cerberus provides powerful yet simple and lightweight data validation functionality out of the box</li> <li>designed to be easily extensible, allowing for custom validation</li> <li>Origin of the name: CERBERUS, n. The watch-dog of Hades, whose duty it was to guard the entrance;</li> </ul> <div class="codehilite"><pre><span></span><code> <span class="n">schema</span> <span class="o">=</span> <span class="p">{</span><span class="s1">&#39;name&#39;</span><span class="p">:</span> <span class="p">{</span><span class="s1">&#39;type&#39;</span><span class="p">:</span> <span class="s1">&#39;string&#39;</span><span class="p">},</span> <span class="s1">&#39;age&#39;</span><span class="p">:</span> <span class="p">{</span><span class="s1">&#39;type&#39;</span><span class="p">:</span> <span class="s1">&#39;integer&#39;</span><span class="p">,</span> <span class="s1">&#39;min&#39;</span><span class="p">:</span> <span class="mi">10</span><span class="p">}}</span> <span class="n">v</span> <span class="o">=</span> <span class="n">Validator</span><span class="p">(</span><span class="n">schema</span><span class="p">)</span> <span class="n">document</span> <span class="o">=</span> <span class="p">{</span><span class="s1">&#39;name&#39;</span><span class="p">:</span> <span class="s1">&#39;Little Joe&#39;</span><span class="p">,</span> <span class="s1">&#39;age&#39;</span><span class="p">:</span> <span class="mi">5</span><span class="p">}</span> <span class="n">v</span><span class="o">.</span><span class="n">validate</span><span class="p">(</span><span class="n">document</span><span class="p">)</span> <span class="c1"># False</span> <span class="n">v</span><span class="o">.</span><span class="n">errors</span> <span class="c1"># {&#39;age&#39;: [&#39;min value is 10&#39;]}</span> </code></pre></div> <p><strong>Follow up and other news</strong></p> <p><strong>Michael:</strong></p> <p><strong>#100DaysOfCode in Python course</strong>: <strong><a href=""></a></strong></p>
Mar 23, 2018
#69 Digging into StackOverflow's 2018 survey results
<h1>Python Bytes 69</h1> <p>Sponsored by DigitalOcean: <strong><a href=""></a></strong></p> <p><strong>Brian #1:</strong> <a href=""><strong>pynb: Jupyter Notebooks as plain Python code with embedded Markdown text</strong></a></p> <ul> <li><code>pynb</code> lets you manage Jupyter notebooks as plain Python code with embedded Markdown text, enabling: <ul> <li><strong>Python development environment</strong>: Use your preferred IDE/editor, ensure style compliance, navigate, refactor, and test your notebooks as regular Python code.</li> <li><strong>Version control</strong>: Track changes, review pull requests and merge conflicts as with regular Python code. The cell outputs are stored separately and don't interfere with versioning.</li> <li><strong>Consistent execution state</strong>: Never lose track again of the execution state. Notebooks are always executed from clean iPython kernels and the cell execution is cached.</li> </ul></li> <li>You also get parameterized notebooks with batch and programmatic execution.</li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>Microsoft’s quantum computing language is now available for</strong></a> <a href=""><strong>m</strong></a><a href=""><strong>acOS</strong></a></p> <ul> <li>New language Q# (<a href="">snippet examples</a>)</li> <li>How do you run a quantum app?</li> <li>Based on topological qubits and quantum computers</li> <li>Now out on macOS &amp; Linux</li> <li>Need to use VS Code (and vs code extension)</li> <li>Comes with Python interoperability (only other language)</li> <li>Also in Jupyter</li> <li>Some real-world examples. See <a href="">this Wired article</a>. <ul> <li><a href="">D-wave</a></li> <li>IBM is making quantum computers commercially available. Since 2016, it has offered researchers the chance to run experiments on a five-qubit quantum computer via the cloud and at the end of 2017 started making its 20-qubit system available online too.</li> </ul></li> </ul> <p><strong>Brian #3: pytest talk in Spanish</strong> </p> <ul> <li>"pytest: recommendations, basic packages for testing in Python and Django"</li> <li>By A. Vallbona (<a href="">@avallbona</a>) From PyConES 2017</li> <li>with <a href="">English slides</a>, and <a href=";">video in Spanish</a>.</li> <li>Some of the topics covered: <ul> <li><code>pytest-django</code></li> <li><code>model-mommy</code> to easily create fixtures based on django models</li> <li><code>pytest-lazy-fixture</code> allows the use the fixtures as parameters to parameterize</li> <li><code>pytest-mock</code>, <code>pytest-cov</code>, <code>pytest-flake8</code></li> <li><code>freezegun</code> to helps us to "freeze" time</li> <li><code>eradicate</code> to eliminate commented code</li> <li><code>pytest-xdist</code> to run tests in parallel</li> </ul></li> </ul> <p><strong>Bonus pytest topic:</strong></p> <ul> <li><a href=""> just added a Reference page</a>, a full reference to pytest’s API.</li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>StackOverflow Developer Survey Results 2018</strong></a></p> <ul> <li>Sample size: Over 100,000 developers</li> <li>55% contribute to open source</li> <li>64% have CS degrees</li> <li>Experience and Belonging <ul> <li>Connection to other devs (increasing over time)</li> <li>Competing with peers (decreasing over time)</li> <li>Not as good as my peers (decreasing over time)</li> </ul></li> <li>How Much Time Do Developers Spend on a Computer? Most: 9-12 hours</li> <li>Python beats C# in usage for the first time</li> <li>Languages: <ul> <li>Most loved: #1 Rust, #2 Kotlin, #3 Python</li> <li>Most dreaded: VB 6 and CoffeeScript</li> <li>Most wanted: #1 Python 25%, #JavaScript 19%, #3 Go 16%</li> </ul></li> <li>Databases: <ul> <li>Loved: PostgreSQL</li> <li>Dreaded: IBM Db2, Memcached, and Oracle</li> <li>Most wanted: MongoDB</li> </ul></li> <li>Editor: VS Code</li> <li>Dev OSes: <ul> <li>Windows: 49%</li> <li>macOS: 27%</li> <li>Linux: 23%</li> </ul></li> </ul> <p><strong>Brian #5:</strong> <a href=""><strong>demoshell</strong></a></p> <ul> <li><a href=""><strong>@</strong></a><a href="">doughellmann</a></li> <li>Doug Hellman (<a href=""><strong>@</strong></a><a href="">doughellmann</a>) <a href="">announces demoshell</a> <ul> <li>Inspired by a tweet from <a href="">@genehack</a> “Hey, speakers, if you're doing live demos in a shell, clear the screen after <em>every</em> <em>command</em> to get the prompt back at the top, so folks in the back can see what you're doing.”</li> </ul></li> <li>demoshell is a simplified shell for live demonstrations. It always shows the command prompt at the top of the screen and pushes command output down instead of letting it scroll up.</li> <li>In his words: “I put it up there to start a discussion. I’d be happy if a bunch of people showed up and wanted to take it over and actually turn it into something useful. I invite people to give it a look. And warn them that too much interest is going to be met with commit privileges on the repo. :-)”</li> </ul> <p><strong>Michael #6: Clear statement on Python 2 EOL</strong></p> <ul> <li>Will there be a period where Py2.7 is in security-only status before hitting EOL?</li> <li>via Nicola Iarocci‏ <a href="">@nicolaiarocci</a> <ul> <li>Yay, @gvanrossum makes it adamantly clear: “Let's not play games with semantics. The way I see the situation for <strong>2.7 is that EOL is January 1st, 2020</strong>, and there will be no updates, <strong>not even source-only security patches</strong>, after that date.” <strong></strong> </li> </ul></li> <li>Support (from the core devs, the PSF, and stops completely on that date.</li> </ul> <p><strong>Follow up and other news</strong></p> <p><strong>Michael:</strong></p> <ul> <li>Eve: MongoDB &amp; Flask-backed RESTful APIs course is out! <ul> <li><strong><a href=""></a></strong></li> </ul></li> <li>Shoutout to everyone I met at <a href="">PyCon Slovakia</a></li> </ul> <p><strong>Brian:</strong> </p> <ul> <li>A couple of recent episodes on Test &amp; Code focusing on project test development: <ul> <li><a href="">What tests to write first</a> </li> <li><a href="">Prioritize software tests with RCRCRC</a></li> <li>Upcoming topics will include beefing up test coverage with things like equivalence partitioning, boundary value analysis, state transition diagrams, state tables, negative testing, …</li> <li>Also learning a lot about developing an open source project and all the tools surrounding that. I’ll discuss those topics in episodes as well.</li> <li>Project used in both episodes, <a href="">cards : a project task tracking / todo list app</a> that will be expanded as I go along talking about different test and software development topics.</li> </ul></li> </ul>
Mar 18, 2018
#68 Python notebooks galore!
<p>Sponsored by DigitalOcean! <strong><a href=""></a></strong></p> <p><strong>Brian #1:</strong> <a href=""><strong>dumb-pypi</strong></a></p> <ul> <li>This takes some fiddling with and trial and error. I definitely need to write up my experiences with this as a blog post.</li> <li>Combine with <code>pip download</code> (covered in <a href="">episode 24</a>), this makes it super easy to create a static locally hosted pypi server, either for all of your packages, or for your proprietary packages.</li> <li>Roughly:</li> </ul> <pre><code> pip download -d my-packages-dir &lt;package name&gt; ls my-packages-dir &gt; package-list.txt dumb-pypi --package-list my-packages-dir \ --packages-url &lt;url of my server&gt; \ --output-dir my-pypi </code></pre> <ul> <li>Now add something like this to requirements.txt or pip commands:</li> <li><code>--trusted-host &lt;my server name&gt; -i http://&lt;my server&gt;/my-pypi/simple</code></li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>Requests-HTML: HTML Parsing for Humans</strong></a></p> <ul> <li>This library intends to make parsing HTML (e.g. scraping the web) as simple and intuitive as possible.</li> <li>When using this library you automatically get: <ul> <li>Full JavaScript support!</li> <li>CSS Selectors (a.k.a jQuery-style, thanks to PyQuery).</li> <li>XPath Selectors, for the faint at heart.</li> <li>Mocked user-agent (like a real web browser).</li> <li>Automatic following of redirects.</li> <li>Connection–pooling and cookie persistence.</li> <li>The Requests experience you know and love, with magical parsing abilities</li> </ul></li> </ul> <p><strong>Brian #3:</strong> <a href=""><strong>A phone number proxy</strong></a></p> <ul> <li>Naomi Pentrel, <a href="">@naomi_pen</a> on twilio blog</li> <li>Set up a phone number that you can share for temporary events to send and receive texts that get forwarded to your actual number.</li> </ul> <p><strong>Michael #4: Notebooks galore part 1:</strong> <a href=""><strong>Datalore</strong></a></p> <ul> <li>In cloud and ready to go</li> <li>Intelligent code editor</li> <li>Out-of-the-box Python tools</li> <li>Collaboration</li> <li>Integrated version control</li> <li>Incremental calculations: Improve and adjust models without hustling with additional recalculations. Datalore follows dependencies between multiple computations and automatically applies relevant recalculations.</li> </ul> <p><strong>Brian #5:</strong> <a href=""><strong>bellybutton</strong></a></p> <ul> <li>by Chase Stevens, <a href="">@hchasestevens</a></li> <li>Tool for creating personal static analysis/style tools like <code>pycodestyle</code>, <code>pylint</code>, and <code>flake8</code></li> <li>Teams often have some of their own style requirements that can’t be expressed as <code>flake8</code> flags and exceptions.</li> <li>Example: deprecating internal library functions and catching that by the linter. </li> </ul> <p><strong>Michael #6:Notebooks galore part 2</strong></p> <ul> <li><a href=";sh=0c41688d">Python 3.6 Jupyter Notebook on Azure</a></li> <li><a href="">Google Colaboratory</a></li> <li><a href="">JupyterLab is Ready for Users</a> <ul> <li>JupyterLab is an interactive development environment for working with notebooks, code and data. Most importantly, JupyterLab has full support for Jupyter notebooks. Additionally, JupyterLab enables you to use text editors, terminals, data file viewers, and other custom components side by side with notebooks in a tabbed work area.</li> </ul></li> <li>you can pip install python packages within python code itself. <ul> <li>Super useful in situation #1 when you need a package that's not included but you don't have access to the shell. </li> <li>If you need to upgrade a package. For example the Pandas version is a little old on Azure, so you can upgrade by simply running:</li> </ul></li> </ul> <pre><code> import pip pip.main(['install', 'pandas', '--upgrade']) </code></pre>
Mar 06, 2018
#67 Result of moving Python to Github
<p>Sponsored by DigitalOcean! <a href="">http://<strong></strong></a></p> <p><strong>Brian #1:</strong> <a href=""><strong>Building a blog with Pelican</strong></a></p> <ul> <li>We did cover Pelican in <a href="">episode 38</a>, but this is a nice tutorial in 7 parts on building a blog.</li> <li>Peter Kazarinoff, <a href="">@pkazarinoff</a> <ul> <li>Nice blog with a focus on engineering students.</li> </ul></li> <li>Starts with installing Python and git and some other tools.</li> <li>Step by step, every action to get a a blog up as a static site hosted on github pages.</li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>Notifiers</strong></a> </p> <ul> <li>Got an app or service and you want to enable your users to use notifications with their provider of choice? </li> <li>Working on a script and you want to receive notification based on its output? </li> <li>A one stop shop for all notification providers with a unified and simple interface.</li> <li>A unified interface means that you already support any new providers that will be added, no more work needed!</li> <li>Some providers <ul> <li>Slack</li> <li>Gmail</li> <li>Telegram</li> <li>Gitter</li> <li>…</li> </ul></li> <li>Python 3 only</li> </ul> <p><strong>Brian #3:</strong> <a href=""><strong>Using Makefiles in Python projects</strong></a></p> <ul> <li>Krzysztof Żuraw, <a href="">@krzysztof_zuraw</a></li> <li>Alerted to this article from kidpixo, <a href="">@kidpixo</a></li> <li>We don’t usually think of Makefiles and Python together, but they can be a handy place to keep common scripts for a project all in one place.</li> <li>This article is a nice gentle intro to Makefiles and shows some cool uses: <ul> <li>cleaning out .pyc files</li> <li>cleaning out egg directories</li> <li>linting</li> <li>running tests with flags</li> <li>starting a test server</li> <li>deploying</li> <li>sorting import files</li> </ul></li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>Result of moving Python to Github</strong></a></p> <ul> <li>See the graph linked in the post</li> <li>A couple of quick numbers (including PRs too) from 2017 compared to 2016: <ul> <li>the number of commit has increased by 190%</li> <li>inserted lines of code has increased by 140%</li> <li>number of unique contributors has increased by 1300%</li> <li>number of returning contributors has increased by 900%</li> </ul></li> <li>One comment was: “Personally, I would like them moving to Gitlab instead, but still good news.” I tend to disagree.</li> </ul> <p><strong>Brian #5:</strong> <a href=""><strong>Self-Deprecation Needs to</strong></a> <a href=""><strong>Stop</strong></a></p> <ul> <li>Maurice Hayward, <a href=""><strong>@</strong></a><a href="">maurice_hayward</a></li> <li>Inspired by some tweets by Stephanie Hurlburt, <a href=""><strong>@</strong></a><a href="">sehurlburt</a></li> <li><strong>Stop saying these words when describing yourself or your accomplishments.</strong> These words are now under BAN: "My project is..." - very small/basic/simple - not that good - a thing I wrote - just by a newbie - something I didn't spend a lot of time/effort on - silly - not that useful Just state the topic and let others be the judge.</li> <li><strong>Really think about the value you bring, then let everybody know.</strong></li> <li><strong>Be Proud of Yourself!</strong></li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>5 speed improvements in Python 3.7</strong></a></p> <ol> <li><strong>Calling methods faster (maybe)</strong> <ol> <li>Python 3.7 adds 2 new Opcodes, LOAD_METHOD and CALL_METHOD for when the compiler sees x.method(...) it uses these new Opcodes.</li> <li>Bound methods with no arguments are now faster</li> </ol></li> <li><strong>str.find() is faster for some characters</strong> <ol> <li>Some unicode characters have an unfortunate issue when scanning a string for occurrences using str.find(x), seeing up to 25x slow down.</li> <li>These are still slower, but now 3x slower than ASCII characters instead of 25x!</li> </ol></li> <li><strong>os.fwalk is 2x faster</strong></li> <li><strong>Regular expressions are faster</strong> <ol> <li>A change was made in Python 3.6 which slowed down this call when flags were passed which were integers. Python 3.7 “fixes” the slowdown but is still not as fast as Python 3.5</li> </ol></li> <li><strong>Regular expressions are faster for case-insensitive matching</strong> <ol> <li>The speed improvement is significant, if you’re matching ASCII characters you can see up to a 20x improvements in matching time since it’s now doing a lookup instead of running lower() over each character.</li> </ol></li> </ol> <p>Follow up and other news</p> <p>Brian: </p> <ul> <li><a href="">Python package maintainers, help test the new PyPI!</a></li> <li><a href="">pytest/pycharm webinar is up</a>.</li> </ul>
Mar 01, 2018
#66 Wait, NoSQL with ACID and transactions?
<p>Sponsored by Rollbar: <strong><a href=""></a></strong></p> <p><strong>Brian #1:</strong> <a href=""><strong>Object-Oriented Programming (OOP) in Python 3</strong></a></p> <ul> <li>Real Python</li> <li>Nice modern introduction to classes, inheritance, and OOP.</li> <li>Classes, objects, attributes, instances, and inheritance.</li> <li>One gotcha not mentioned <ul> <li>The <code>__init__()</code> method of a base class is not called automatically by derived classes. If you override it, you need to call <code>super().__init__()</code>.</li> </ul></li> <li>Also, check out <a href="">attrs</a> for much of our OOP needs</li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>ScriptedForms</strong></a></p> <ul> <li>Quickly create live-update GUIs for Python packages using Markdown and a few custom HTML elements. </li> <li>Just write in markdown + variables / UI types</li> <li>Based on Jupyter</li> </ul> <p><strong>Brian #3:</strong> <a href=""><strong>MongoDB to add multi-document transactions and ACID</strong></a></p> <ul> <li>Mind blown. Didn’t see this coming</li> <li>“MongoDB 4.0 will add support for multi-document transactions, making it the only database to combine the speed, flexibility, and power of the document model with ACID data integrity guarantees. Through snapshot isolation, transactions provide a globally consistent view of data, and enforce all-or-nothing execution to maintain data integrity.”</li> <li>Due out this summer.</li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>Python packaging pitfalls</strong></a></p> <ul> <li>Just a short list of packaging blunders <ul> <li>Forgetting to clean the build dir</li> <li>Forgetting to specify package data</li> <li>Fine grained</li> <li>Using package_data, or worse: fine grained package_data</li> <li>Listing excludes/prunes before includes/grafts</li> <li>Hardcoding packages list in</li> <li>Hardcoding py_modules list in</li> <li>Importing your package in</li> <li>Importing unavailable tools in</li> <li>Messing with the environment</li> <li>Your tests do not test the installed code</li> </ul></li> </ul> <p><strong>Brian #5:</strong> <a href=""><strong>Blogging principles</strong></a></p> <ul> <li>Julia Evans @b0rk</li> <li>Be honest about what you know</li> <li>Try not to write anything too long. <ul> <li>(My note: don’t shy away from long tutorials. Just don’t only do long stuff)</li> </ul></li> <li>Be positive.</li> <li>Write for the past you. </li> <li>Stick with your own experience.</li> <li>It’s ok if not everyone likes it <ul> <li>Don’t try to keep one upping yourself.</li> </ul></li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>pipenv is officially official</strong></a></p> <ul> <li>Pipenv — the officially recommended Python packaging tool from, free (as in freedom). <ul> <li>Pipenv is a tool that aims to bring the best of all packaging worlds (bundler, composer, npm, cargo, yarn, etc.) to the Python world. </li> <li>Windows is a first–class citizen, in our world.</li> </ul></li> <li>Benefits? <ul> <li>It automatically creates and manages a virtualenv for your projects</li> <li>adds/removes packages from your Pipfile as you install/uninstall packages</li> <li>generates the ever–important Pipfile.lock, which is used to produce deterministic builds.</li> </ul></li> </ul> <p>Follow up and other news</p> <p><strong>Brian</strong></p> <ul> <li>Productive pytest with PyCharm webinar was recorded Thursday 22nd of Feb. </li> <li>Will be available here: <a href=""><strong></strong></a></li> </ul> <p>Michael:</p> <ul> <li>Embed Python in Unreal Engine 4 <strong><a href=""></a></strong></li> <li>Pandas documentation sprint <strong><a href=""></a></strong></li> </ul>
Feb 23, 2018
#65 Speed of your import statements affecting performance?
<p>Sponsored by Rollbar: <a href=""></a></p> <p><strong>Brian #1:</strong> <a href=""><strong>pygal</strong></a> <strong>: Simple Python Charting</strong> </p> <ul> <li>Output SVG or PNG</li> <li>Example Flask App (also django response) part of documentation.</li> <li>Enough other bits of doc to get you a chart in a web page super fast.</li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>Thoughts on becoming a self-taught programming</strong></a></p> <ul> <li>Basic format:</li> <li>I'm 31 days into self-studying Python and am loving every minute of it!</li> <li>A few questions: <ul> <li>What were you doing before you began self-studying programming?</li> <li>What made you want to study programming on your own?</li> <li>How did you start (which resources and language)?</li> <li>How long did it take for you to feel confident enough in your skills and knowledge to know you could be employed as a programmer?</li> <li>What else did you do besides self-study that helped you in your journey to becoming a programmer?</li> <li>What's next for you?</li> </ul></li> </ul> <p><strong>Brian #3:</strong> <a href=""><strong>How to speed up Python application startup time (timing imports in 3.7)</strong></a></p> <ul> <li>Python 3.7 includes <code>-X importtime</code> option that allows you to profile the time it takes to do all the imports.</li> <li>Way cool tool to help optimize the startup time of an application.</li> </ul> <p><strong>Michael #4:</strong> <a href="">AnPyLar - The Python web front-end framework</a></p> <ul> <li>Create web applications with elegance, simplicity and yet full power with Python and components</li> <li>MISSION: Empower all Python programmers to work not only on the back-end but also on the front-end with the same language of choice</li> <li>Features <ul> <li>Reactive programming and Promises</li> <li>Python standard formatting as templates</li> <li>reusable components</li> <li>Scoped styling for component</li> <li>Integrated routing engine</li> </ul></li> </ul> <p><strong>Brian #5:</strong> <a href=""><strong>Migrating to Python 3 with pleasure</strong></a></p> <ul> <li><strong>“A short guide on features of Python 3 for data scientists”</strong></li> <li>Quick tutorial through examples of <code>pathlib</code>.</li> <li>Type hinting and how cool it works with editors (PyCharm example shown)</li> <li>Adding runtime type enforcement for specific methods using <a href="">enforce</a></li> <li>Using function annotations for units, as done in <a href="">astropy</a>. </li> <li>Matrix multiplication with <code>@</code>.</li> <li>Globbing with <code>**</code>. <ul> <li><code>found_images = glob.glob('/path/**/*.jpg', recursive=True)</code></li> </ul></li> <li>Also … underscores in numeric literals, f-strings, true division with <code>/</code>, integer division with <code>//</code>, and lots of more fun goodies.</li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>Moving to Python 3</strong></a></p> <ul> <li>Many of these issues were corrected just by running 2to3, which not only fixed many of the compatibility issues <ul> <li>Outdated external libraries which needed to be updated to newer versions featuring Python 3 compatibility <code>basestring</code> to <code>str</code>, <code>urlparse</code> to <code>urllib.urlparse</code> and similar major changes</li> <li>Dictionary change like <code>iteritems()</code> to <code>items()</code>, or <code>.items()</code> now returning a view.</li> <li>Things that weren't needed anymore, like Django's <code>force_unicode</code> or <code>__future__</code> library tools.</li> </ul></li> <li>Once we finished working on the "low-hanging fruits", the next step was to run Aphrodite's test suite and achieve zero errors.</li> <li>Lessons learned <ul> <li>Code coverage was originally around 70%,</li> <li>Keeping the Python 3 branch up to date with master</li> <li>A non-trivial feature was delivered during the migration (via feature branch)</li> <li>The pickle protocol version in python 3 can be higher than the highest available in Python 2.7. So we needed to add versioning to our Django caches</li> <li>Each modified file had to comply with flake8 linting rules</li> </ul></li> <li>Afrodita is currently running on Google App Engine Flexible, and one of the features our team loves with is traffic splitting</li> <li>With this feature, we can do <a href="">canary releases</a> with ease: We just deploy our new version of the service, and start redirecting small amounts of traffic traffic while we monitor for unexpected errors.</li> <li>After some minor bugfixes, we could bring the traffic of the Python 3.6 version to 100% with confidence. We also had the old version available for instant rollback, thanks to how parallel versions and traffic splitting work in GAE flexible.</li> </ul> <p><strong>Our news</strong></p> <p>Brian:</p> <ul> <li>Upcoming webinar: <a href="">Productive pytest with Pycharm</a></li> </ul> <p>Michael:</p> <ul> <li>My GUI example: <a href=""></a></li> </ul>
Feb 14, 2018
#64 The GUI phoenix rises with wxPython
<p>Sponsored by DigitalOcean: <strong><a href=""></a></strong></p> <p><strong>Brian #1: wxPython 4,</strong> <strong>Pheonix is now live and supports Python 3</strong></p> <ul> <li><a href="">wxPython on PyPI</a></li> <li><a href="">4.0.0</a>, <a href="">4.0.1 release notes</a></li> <li>If you haven’t played with wxPython for a while, now might be a good time.</li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>typeshed</strong></a></p> <ul> <li>Typeshed contains external type annotations for the Python standard library and Python builtins, as well as third party packages.</li> <li>This data can e.g. be used for static analysis, type checking or type inference. </li> <li>Used as the basis of mypy and PyCharm’s magic</li> <li>Each Python module is represented by a .pyi "stub". This is a normal Python file (i.e., it can be interpreted by Python 3), except all the methods are empty. Python function annotations (PEP 3107) are used to describe the types the function has.</li> <li>Here’s what one of these exeternal definitions looks like:</li> </ul> <pre><code> class NodeVisitor(): def visit(self, node: AST) -&gt; Any: ... def generic_visit(self, node: AST) -&gt; None: ... </code></pre> <p><strong>Brian #3:</strong> <a href=""><strong>Coverage 4.5 adds configurator plug-ins</strong></a></p> <ul> <li>“There’s one new feature: <a href="">configurator plug-ins</a>, that let you run Python code at startup to set the configuration for coverage. This side-steps a requested feature to have different exclusion pragmas for different versions of Python.”</li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>Python integrated into Unreal Engine</strong></a></p> <ul> <li>via Pirogov Alexander‏ ( <a href="">@Pie_Daddy</a> )</li> <li>tl;dr: Autodesk university plans to integrate Python into Unreal Engine for the data integration pipeline and ease the process of bringing assets into the game.</li> <li>Autodesk is working on that will solve complicated problems with bringing CAD data into the Unreal Engine.</li> <li>Where they are today: <ul> <li>The Datasmith workflow toolkit, currently in beta, makes moving data into Unreal Engine as frictionless as possible. </li> <li>Datasmith provides high-fidelity translation of common scene assets such as geometry, textures, materials, lights and cameras from popular DCC and CAD applications into Unreal Engine.</li> </ul></li> </ul> <p><strong>Brian #5:</strong> <a href=""><strong>Python 3.7.0b1</strong></a> <strong>: Beta means we should be testing it!!!</strong></p> <ul> <li>If not people like us and our listeners, then who? Seems like we’re a good set of beta testers.</li> <li>What are you going to test?</li> <li>I'm going to look at breakpoint() and data classes.</li> </ul> <p><strong>Michael #6: Releases abound!</strong></p> <ul> <li>Django security releases issued: 2.0.2 and 1.11.10 <a href=""></a></li> <li>Python 3.4.8 (security) <a href=""></a></li> <li>Python 3.5.5 (security) <a href=""></a> <ul> <li>libexpat XML lib had a security issue</li> </ul></li> <li>Channels 2.0 is a major rewrite of Channels <ul> <li>See <a href="">Talk Python’s interview</a> for more details</li> <li>Notably: Python 2.7 and 3.4 are no longer supported.</li> </ul></li> </ul> <p><strong>Our news</strong></p> <p>Brian:</p> <ul> <li>Speaking at PyCon 2018. “PyCharm and pytest”. Speaking with Paul Everitt</li> <li>Upcoming webinar: <a href="">Productive pytest with Pycharm</a> <ul> <li>Feb 22, registration open</li> </ul></li> </ul> <p>Michael:</p> <ul> <li><a href=""><strong>Webinar Recording:</strong></a> “MongoDB Quickstart with Python and PyCharm” with Michael Kennedy</li> </ul>
Feb 09, 2018
#63 We're still on a desktop GUI kick
<p>Sponsored by DigitalOcean: <strong><a href=""></a></strong></p> <p><strong>Brian #1:</strong> <a href=""><strong>A brief tour of Python 3.7 data classes</strong></a></p> <ul> <li>a great write-up of the upcoming data classes via Anthony Shaw</li> <li>“Data classes are a way of automating the generation of boiler-plate code for classes which store multiple properties. They also carry the benefit of using Python 3’s new type hinting.”</li> <li>Default magic methods <ul> <li>In the default setting, any dataclass will implement <code>__init__</code>, <code>__repr__</code>, <code>__str__</code> and <code>__eq__</code> for you.</li> <li>The <code>__init__</code> method will have <strong>keyword-arguments</strong> with the same type annotations that are specified on the class.</li> <li>The <code>__eq__</code> method will compare all dataclass attributes in order.</li> <li>All fields are declared at the top of the class and type hinting is <strong>required</strong>.</li> </ul></li> <li>Also covered <ul> <li>type hinting</li> <li>mutability (and frozen)</li> <li>customizing the fields</li> <li>post-init processing : optional <code>__``*post_init_*``_</code> will run after the generated <code>_``*_init_*``_</code></li> <li>inheritance</li> </ul></li> </ul> <p><strong>Michael #2:</strong> SQLite [The Databaseology Lectures - CMU Fall 2015]</p> <ul> <li>Lots of DBs covered here: <strong><a href=""></a></strong></li> <li>SQLite at this <a href=";index=2&amp;list=PLSE8ODhjZXjakeQR57ZdN5slUu2oPUr1Y"><strong>YouTube video</strong></a></li> </ul> <p><strong>Brian #3:</strong> <a href=""><strong>dryable</strong></a> <a href=""><strong>:</strong></a> <a href=""><strong>a useful dry-run decorator for python</strong></a></p> <ul> <li>short circuit methods within your project during dry runs.</li> <li>example shows how to add a command line flag <code>--dry-run</code>.</li> <li>The test code is useful for understanding it also.</li> <li><p>Example import dryable</p> <p>@dryable.Dryable('foo') def return_something(): return 'something'</p></li> </ul> <p></p> <pre><code>from something import return_something import dryable def test_normal_return(): dryable.set(False) assert return_something() == 'something' def test_dry_return(capsys): dryable.set(True) assert return_something() == 'foo' </code></pre> <p><strong>Michael #4:</strong> </p> <ul> <li>These are some pretty cool examples. <ul> <li><a href=""></a></li> <li><a href=""></a></li> <li><a href=""></a></li> </ul></li> </ul> <p><strong>Brian #5:</strong> <a href=""><strong>PEP Explorer - Explore Python Enhancement Proposals</strong></a></p> <ul> <li>Cool idea. Might need some work though. I can’t find any accepted PEPs for 3.7, including <a href="">557, data classes</a>.</li> <li>I’m ok with giving Anthony some shade on this, as we highlighted his writing in the first item.</li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>TKInter Tutorial</strong></a></p> <ul> <li>via @likegeeks</li> <li>Create your first GUI application</li> <li>Create a label and button widgets</li> <li>Input and combo boxs, menus, progressbars and more</li> </ul> <h2>Our news</h2> <p>Michael</p> <ul> <li>I built something with <a href="">Gooey</a> this weekend, it was wonderful.</li> <li>Self-serve team purchases and discounts at Talk Python Training</li> </ul>
Feb 01, 2018
#62 Wooey and Gooey are simple Python GUIs
<p>Brought to you by Datadog <a href=""></a></p> <p><strong>Brian #1:</strong> <strong>Dan Bader takes over Real Python</strong></p> <ul> <li><a href="">Announcement email</a>, with what Michael, Fletcher, and Jeremy are doing now</li> <li>Dan is on the show and tells us all about it.</li> </ul> <p><strong>Michael #2: Still more Python GUIs</strong></p> <ul> <li><strong><a href=""></a></strong> <ul> <li>A Django app that creates automatic web UIs for Python scripts.</li> <li>Wooey is a simple web interface to run command line Python scripts. Think of it as an easy way to get your scripts up on the web for routine data analysis, file processing, or anything else.</li> <li>Wooey was envisioned as a system to allow data analysts to be able to easily: <ul> <li>Autodocument workflows for data analysis (simple model saving).</li> <li>Enable fellow co-workers with no command line experience to utilize python scripts.</li> <li>Enable the easy wrapping of any program in simple python instead of having to use language specific to existing tools such as Galaxy.</li> </ul></li> <li>Try the demo server: <a href=""></a></li> </ul></li> <li><strong><a href=""></a></strong> <ul> <li>Turn (almost) any Python command line program into a full GUI application with one line</li> <li>See the <a href=""><strong>screenshots here</strong></a></li> <li>Gooey converts your Console Applications into end-user-friendly GUI applications. </li> <li>It lets you focus on building robust, configurable programs in a familiar way, all without having to worry about how it will be presented to and interacted with by your average user.</li> </ul></li> <li>And Toga: <a href=""></a></li> </ul> <p><strong>Brian #3:</strong> <a href=""><strong>Python’s misleading readability</strong></a></p> <ul> <li>Ned Batchelder</li> <li><code>is</code> and <code>or</code> are not obvious and can confuse people new to the language, new to programming. <ul> <li><code>1000 + 1 is 1001</code> → <code>1000 + 1 == 1001</code></li> <li><code>answer == "y" or "yes``"</code> → <code>answer in {"y", "yes"}</code></li> </ul></li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>warp2 access</strong></a> </p> <ul> <li>python2 code from python3</li> <li>It communicates with the subprocess using pickle, so there are limitation to using it - if you need to send unpicklable data, that’s a problem.</li> </ul> <p><strong>Brian #5:</strong> <strong>Help! My tests can’t see my code!</strong></p> <ul> <li>Probably should be an episode on <a href="">Test &amp; Code</a>, and maybe I’ll do that also, but it’s a big enough roadblock to many newcomers to <a href="">pytest</a>, that I want to get the word out on how to fix it.</li> <li>A best practice is to put your test code in a folder called tests.</li> <li>Now, if you are sitting in the parent directory, where you can see both your modules/packages under test and the <code>tests</code> directory, and you run <code>pytest</code>, your test code has to have some way to import the code under test.</li> <li>If you are in a hurry. Homework due in an hour, project manager breathing down your neck, or whatever, then there are two easy options: <ul> <li><code>python -m pytest</code></li> <li><code>python</code> adds the current directory where you start it to <code>PYTHONPATH</code>, <code>pytest</code> does not.</li> <li><code>pip install pytest-pythonpath</code></li> <li><a href=""></a></li> <li>This plugin adds the current directory to <code>PYTHONPATH</code>, and adds some hooks that let you define search paths in your <code>pytest.ini</code> file.</li> </ul></li> <li>When you have time.. <ul> <li>Create a <code></code> file for your code. And…</li> <li><code>pip install -e ./your_project</code></li> <li>This allows you to continue working on your code while letting your test code see the code under test</li> <li>This method is friendlier to <code>tox</code>.</li> </ul></li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>Cement - Framework for CLI</strong></a></p> <ul> <li>Cement is an advanced CLI Application Framework for Python. </li> <li>Its goal is to introduce a standard, and feature-full platform for both simple and complex command line applications </li> <li>Also supports rapid development needs without sacrificing quality.</li> <li>Core features <ul> <li>Core pieces of the framework are customizable via handlers/interfaces</li> <li>Extension handler interface to easily extend framework functionality</li> <li>Config handler supports parsing multiple config files into one config</li> <li>Argument handler parses command line arguments and merges with config</li> <li>Log handler supports console and file logging</li> <li>Plugin handler provides an interface to easily extend your application</li> <li>Hook support adds a bit of magic to apps and also ties into framework</li> <li>Handler system connects implementation classes with Interfaces</li> <li>Output handler interface renders return dictionaries to console</li> <li>Cache handler interface adds caching support for improved performance</li> </ul></li> </ul> <p>Our news</p> <p>Michael</p> <ul> <li>Conferences! <ul> <li>PyCascades in Vancouver BC on Jan 22, 23. Was great, get to it next year.</li> <li><a href="">PyColumbia</a>, February 9, 10 and 11 Medellin, Colombia - I won't be there but if you are able to make it get your tickets</li> <li>PyCon Slovakia, March 9-11 in Bratislava. I'll be speaking there and doing a workshop. </li> <li>pycon us: Cleveland OH May 10th. I just finalized all my travel plans. I hope to see you there, please stop by our booth.</li> <li>PyCarribian: Santo Domingo, Dominican Republic, 17-18 February, 2018</li> </ul></li> <li>Podcast <a href=""></a></li> </ul>
Jan 26, 2018
#61 On Being a Senior Engineer
<p>Sponsored by DigitalOcean: <strong><a href=""></a></strong></p> <p><strong>Brian #1:</strong> <a href=""><strong>PEP 412's dict key sharing for classes</strong></a></p> <ul> <li>"memory use is reduced by 10% to 20% for object-oriented programs with no significant change in memory use for other programs."</li> <li>To benefit from this, make sure all attributes used in life of class instances are initialized within <code>__init__()</code>. </li> <li>Video from PyCon 2017 <ul> <li><a href="">Brandon Rhodes The Dictionary Even Mightier PyCon 2017</a></li> <li>Look at description at about 14 minutes on in the video</li> </ul></li> <li>Suggested by <a href="">Ned Letcher</a></li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>Python Hunter</strong></a></p> <ul> <li>via Ivan Pejić</li> <li>Hunter is a flexible code tracing toolkit, not for measuring coverage, but for debugging, logging, inspection and other nefarious purposes. It has a Python API, terminal activation (see Environment variable activation). and supports tracing other processes (see Tracing processes).</li> <li>The default action is to just print the code being executed</li> <li>Based on cython</li> </ul> <p><strong>Brian #3:</strong> <a href=""><strong>Ten Things I Wish I’d Known About bash</strong></a></p> <ul> <li>I started with ksh on Solaris/HP-UX, used zsh for few years.</li> <li>Mostly now, I use bash, because it’s everywhere. Mac/Windows/Linux</li> <li>For windows: <a href=""><strong>git for windows</strong></a> <ul> <li>Even if you don't need git, git for windows comes with fully integrated unix tools and bash and it just works as you expect.</li> <li>you can launch windows applications</li> <li>most of the frequent bash commands are there</li> <li>If you really don’t want bash, consider <a href=""><strong>cmder</strong></a></li> </ul></li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>Snakefooding Python Code For Complexity Visualization</strong></a></p> <ul> <li><a href="">Snakefood</a> is a tool written by Martin Blais to create Python dependency graphs. </li> <li>Combined with GraphViz, snakefood can create beautiful visualizations of Python codebases.</li> <li>Python Web Frameworks: The different development philosophies of Bottle, Django, Flask, and Pyramid are apparent by looking at their snakefood graphs.</li> <li>Bottle: A fast and simple micro framework for Python web applications.</li> <li>Django: A batteries-included web framework for perfectionists with deadlines.</li> <li>Flask: A microframework for Python.</li> <li>Pyramid: A small, fast, down-to-earth, open source Python web framework. It makes real-world web application development and deployment more fun, more predictable, and more productive.</li> <li>Also Queueing Implementations</li> </ul> <p><strong>Brian #5:</strong> <a href=""><strong>On Being a Senior Engineer</strong></a></p> <ul> <li>2012 article that's still very valid</li> <li>Obligatory Pithy Characteristics of Mature Engineers</li> <li>Mature engineers ... <ul> <li>seek out constructive criticism of their designs.</li> <li>understand the non-technical areas of how they are perceived.</li> <li>do not shy away from making estimates, and are always trying to get better at it.</li> <li>have an innate sense of anticipation, even if they don’t know they do.</li> <li>understand that not all of their projects are filled with rockstar-on-stage work.</li> <li>lift the skills and expertise of those around them.</li> <li>make their trade-offs explicit when making judgements and decisions.</li> <li>don’t practice CYAE (“Cover Your Ass Engineering”)</li> <li>are empathetic.</li> <li>don’t make empty complaints.</li> <li>are aware of cognitive biases: <ul> <li>Self-Serving Bias</li> <li>Fundamental Attribution Error</li> <li>Hindsight Bias</li> <li>Outcome Bias</li> <li>Planning Fallacy</li> </ul></li> </ul></li> <li>The Ten Commandments of Egoless Programming <ol> <li>Understand and accept that you will make mistakes.</li> <li>You are not your code.</li> <li>No matter how much “karate” you know, someone else will always know more. </li> <li>Don’t rewrite code without consultation. </li> <li>Treat people who know less than you with respect, deference, and patience. </li> <li>The only constant in the world is change. </li> <li>The only true authority stems from knowledge, not from position. </li> <li>Fight for what you believe, but gracefully accept defeat. </li> <li>Don’t be “the coder in the corner.” </li> <li>Critique code instead of people – be kind to the coder, not to the code.</li> </ol></li> <li>also: <ul> <li>Novices versus Experts </li> <li>Dirty secret: mature engineers know the importance of (sometimes irrational) feelings people have. (gasp!)</li> <li>“It is amazing what you can accomplish if you do not care who gets credit.”</li> </ul></li> </ul> <p><strong>Michael #6: Python UI frameworks</strong></p> <ul> <li>TkInter (<a href="">tutorial</a>) - not amazing, not at all (<a href="">example</a>).</li> <li><a href="">PySide</a> and Qt - hard to install, weird licensing and versioning, but has a nice designer</li> <li>Kivy and PyGame/PyOpenGL - game / simulation like</li> <li><a href="">wxPython</a> seems not bad actually <ul> <li><a href="">example</a></li> <li><a href="">widgets</a></li> <li><a href="">wxFormBuilder</a> - a RAD tool for wxWidgets GUI design</li> <li><a href="">wxGlade</a> is a GUI designer</li> </ul></li> <li>What else? A few platform specific examples</li> <li>The problem: was discussed last week</li> <li>Some more Electron.JS like solutions</li> <li><a href=""><strong></strong></a> <ul> <li>Eel is a little Python library for making simple Electron-like offline HTML/JS GUI apps, with full access to Python capabilities and libraries.</li> <li>It hosts a local webserver, then lets you annotate functions in Python so that they can be called from Javascript, and vice versa.</li> </ul></li> <li><a href="">CEFPython</a> - Chrome browser control, a HTML 5 based Python GUI framework.</li> </ul>
Jan 16, 2018
#60 Don't dismiss SQLite as just a starter DB
<p>Brought to you by Datadog <a href=""></a></p> <p><strong>Brian #1:</strong> <a href=";"><strong>Who's at nine?</strong></a></p> <ul> <li>Organic Idiocy</li> <li>Inspired by <a href="">Michael talking about programming Alexa in episode 33</a> <ul> <li><a href="">Twitter thread</a></li> </ul></li> <li>Using <ul> <li><a href="">Flask-Ask</a> for Alexa</li> <li><a href="">Flask-Assistant</a> for Google Home</li> </ul></li> <li><a href="">Talk Python 146</a> is all about Flask Ask and Assistant this week. ;)</li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>Retiring Python as a teaching language</strong></a></p> <ul> <li>Why did he write this? <ul> <li>Then one day a student will innocently ask "Instead of running the poker simulator from the command line, how can I put it in a window with a button to deal the next hand?"</li> </ul></li> <li>The ensuing Twitter conversation was very interesting. Scroll this status, it’s pretty comprehensive <a href=""></a></li> </ul> <p><strong>Brian #3:</strong> <strong>Don't dismiss SQLite as just a starter DB</strong></p> <ul> <li>SQLite is a single file db that <a href="">comes with Python</a>.</li> <li>A listener pointed us to a couple cool things about SQLite</li> <li>A great interview with the developer <a href="">The Changelog, episode 201</a>.</li> <li>It's extensive documentation on how <a href="">SQLite is tested</a>.</li> <li>Of course, for web applications and other applications that have to deal with extreme concurrency, you need a client server database </li> <li>Many applications don't have extreme concurrency needs.</li> <li>Sticking with SQLite might be just fine for quite a long time for many apps.</li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>Chalice: Python Serverless Microframework for AWS</strong></a></p> <ul> <li>Chalice is a python serverless microframework for AWS. It allows you to quickly create and deploy applications that use Amazon API Gateway and AWS Lambda. </li> <li>It provides: <ul> <li>A command line tool for creating, deploying, and managing your app</li> <li>A familiar and easy to use API for declaring views in python code (Flask)</li> <li>Automatic IAM policy generation</li> </ul></li> <li>Compare to Zappa: <a href=""></a></li> </ul> <p><strong>Brian #5:</strong> <a href=""><strong>Fastest way to uniquely a list in Python &gt;=3.6</strong></a></p> <ul> <li>Nice analysis of different ways to uniquify a list.</li> <li>Punchline: <ul> <li>The fastest way to uniqify a list of hashable objects (basically immutable things) is: <code>list(set(seq))</code></li> <li>And the fastest way, if the order is important is: <code>list(dict.fromkeys(seq))</code></li> </ul></li> </ul> <p><strong>Michael #6: PyTexas and PyCon AU vidoes are up</strong></p> <ul> <li><a href="">PyTexas</a> <ul> <li>Notable PyTexas videos <ul> <li>Micropython</li> <li>What is ML?</li> <li>C for yourself</li> <li>Python and .NET</li> </ul></li> </ul></li> <li><a href="">PyCon AU</a> <ul> <li>Notable PyCon AU videos <ul> <li>Gradual typing</li> <li>Hot reloading Python web-servers at scale </li> <li>Prototyping Python Microservices in Production </li> <li>Secrets of a WSGI master. </li> <li>Python 3 for People Who Haven't Been Paying Attention </li> <li>Identity 2.0: the what, why and how of social and federated login </li> <li>Python: Ludicrous mode (with Django) </li> <li>Scaling Down: Running Large Sites Locally </li> </ul></li> </ul></li> </ul> <p><strong>Our news</strong></p> <p>Michael</p> <p><a href=""><strong>Mastering PyCharm is out</strong></a>. Includes</p> <ul> <li>Learn to manage Python projects in PyCharm (large and small)</li> <li>Create web applications (Pyramid, Flask, Django, and more)</li> <li>Use PyCharm's special data science mode</li> <li>Refactor your Python code with confidence</li> <li>Learn about code smells and duplicate code tooling</li> <li>Access git, github, and use git flow</li> <li>Use the visual debugger to understand code flow and state</li> <li>Make your code more reliable with unit testing and pytest</li> <li>Create new Python packages</li> <li>And lots more</li> </ul> <p>Webcast with JetBrains: <a href=""><strong>MongoDB Quickstart with Python and PyCharm</strong></a> <strong>Jan 30</strong></p>
Jan 11, 2018
#59 Instagram disregards Python's GC (again)
<p>Sponsored by DigitalOcean: <strong><a href=""></a></strong></p> <p><strong>Brian #1:</strong> <strong>gc.freeze() and Copy-on-write friendly Python garbage collection</strong></p> <ul> <li><a href="">Copy-on-write friendly Python garbage collection</a> - Instagram</li> <li><a href="">gc.freeze() now part of Python 3.7</a> - github pull request</li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>SpeechPy - A Library for Speech Processing and Recognition</strong></a></p> <ul> <li>A Library for Speech Processing and Recognition</li> <li>More foundation for data science than shooting out words.</li> <li>Based on MFCC (Mel Frequency Cepstral Coefficient) <ul> <li>The first step in any automatic speech recognition system is to extract features i.e. identify the components of the audio signal that are good for identifying the linguistic content and discarding all the other stuff which carries information like background noise, emotion etc. </li> </ul></li> <li>Citation section is a nice touch</li> </ul> <p><strong>Brian #3:</strong> <a href=""><strong>PyBites Code Challenges : Bites of Py</strong></a></p> <ul> <li>Like code katas, coding challenges you can do on your own.</li> <li>“Bites of Py are self contained 20-60 min Python (3.6) code challenges you can code and verify in the browser.”</li> <li>Use pytest to check answers</li> <li>See pytest output so you can partially solve challenges and see where it fails.</li> <li>BTW, min() takes a key, like sort() and sorted(). I learned that this morning.</li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>How big is the Python Family</strong></a></p> <ul> <li>CPython, Jython, IronPython, Python for .NET, Cython, PyPy, MicroPython, and recently Grumpy</li> <li>This is why I don’t like the word “Python interpreter” but rather use “Python runtime” even though it’s less common.</li> </ul> <p><strong>Brian #5:</strong> <a href=""><strong>Dramatiq: simple task processing</strong></a></p> <ul> <li><a href="">Interview on Podcast.init</a></li> <li><a href="">Cookbook</a> included in documentation to get started pretty quick.</li> <li>Inspired by <a href="">Celery</a>, but probably a bit easier to get into if you are new to task processing.</li> <li>License is interesting</li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>Controlling Python Async Creep</strong></a></p> <ul> <li>From friend of the show Cristian Medina</li> <li>Boundary between sync and async can get tricky</li> <li>The complication arises when invoking awaitable functions. Doing so requires an async defined code block or coroutine. A non-issue except that if your caller has to be async, then you can’t call it either unless its caller is async. Which then forces its caller into an async block as well, and so on. This is “async creep”.</li> <li>Solutions or techniques <ul> <li>Waiting for blocks of async code <ul> <li>The general guideline is to start with things that wait on I/O, like file or socket access, HTTP requests, etc.</li> <li>Once you know which pieces to optimize, start identifying the ones that can run on top of each other.</li> <li>Nice example using a web service</li> </ul></li> <li>Use a thread <ul> <li>Next example creating a dedicated asyncio loop in the secondary thread</li> </ul></li> <li>Mixing sync and async <ul> <li>Let’s look at something more complicated. What if you have a library or module where most functions can run in parallel, but you only want to do so if the caller is async?</li> <li>This could prove useful to any python packages that are wanting to add support for asynchronous execution while still supporting legacy code. </li> </ul></li> </ul></li> </ul> <p><strong>Extra (michael)</strong>: The <strong>PyTennessee conference</strong> will be held February 10-11, 2018. We recently announced our schedule (<a href=""></a>), and tickets are on sale now (<a href=""></a>). A smaller, regional conference is a great way to meet people, make new Python friends, and hear some great talks without having to fight the crowds of the larger conferences.</p> <p>If anyone wants to buy a ticket and wants a 10% discount, they can use the code <code>PythonBytes</code> during checkout.</p> <p><strong>In the news</strong></p> <ul> <li>Not much to do about this but pay attention: A critical design flaw in virtually all microprocessors allows attackers to dump the entire memory contents off of a machine/mobile device/PC/cloud server etc. <ul> <li><a href=""></a></li> <li><a href=""></a></li> <li>Probably excellent coverage on</li> </ul></li> <li>From NY Times: <ul> <li><em>The two problems, called Meltdown and Spectre, could allow hackers to steal the entire memory contents of computers, including mobile devices, personal computers and servers running in so-called cloud computer networks.</em></li> <li><em>There is no easy fix for Spectre, which could require redesigning the processors, according to researchers. As for Meltdown, the software patch needed to fix the issue could slow down computers by as much as 30 percent — an ugly situation for people used to fast downloads from their favorite online services.</em></li> </ul></li> </ul> <p><strong>Our news</strong></p> <p>Michael: <strong>Everything Bundle</strong>: <strong><a href=""></a></strong></p> <p>Includes Mastering PyCharm, Python 3: An Illustrated Tour, Intro to Ansible, and much more.</p>
Jan 05, 2018
#58 Better cache decorators and another take on type hints
<p>Sponsored by DigitalOcean: <strong><a href=""></a></strong></p> <p><strong>Brian #1:</strong> <a href=""><strong>Instagram open sources MonkeyType</strong></a></p> <ul> <li><em>Carl Meyer, an engineer on Instagram’s infrastructure team.</em></li> <li>(Note: <a href="">we talked about Dropbox’s pyannotate in episode 54</a>. pyannotate is not on Python3 yet and generates comment style annotations that are Py2 compatible)</li> <li>MonkeyType is Instagram’s tool for automatically adding type annotations to your Python 3 code via runtime tracing of types seen.</li> <li>Requires Python 3.6+</li> <li>Generates only Python 3 style type annotations (no type comments)</li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>cachetools</strong></a></p> <ul> <li>Extensible memoizing collections and decorators</li> <li>Think variants of Python 3 Standard Library @lru_cache function decorator</li> <li>Caching types: <ul> <li><code>cachetools.Cache</code> Mutable mapping to serve as a simple cache or cache base class.</li> <li><code>cachetools.LFUCache</code> Least Frequently Used (LFU) cache implementation</li> <li><code>cachetools.LRUCache</code> Least Recently Used (LRU) cache implementation</li> <li><code>cachetools.TTLCache</code> LRU Cache implementation with per-item time-to-live (TTL) value.</li> <li>And more</li> </ul></li> <li>Memoizing decorators <ul> <li><code>cachetools.cached</code> Decorator to wrap a function with a memoizing callable that saves results in a cache.</li> <li>Note that cache need not be an instance of the cache implementations provided by the <code>cachetools</code> module. cached() will work with any mutable mapping type, including plain dict and <code>weakref.WeakValueDictionary</code>.</li> <li>Can pass key function for hash insertions and lock object for thread safety.</li> </ul></li> </ul> <p><strong>Brian #3:</strong> <a href=""><strong>Going Fast with SQLite and Python</strong></a></p> <ul> <li>Charles Leifer</li> <li>Many projects start with SQLite, as it’s distributed with Python as <a href="">sqlite3</a>.</li> <li>This article discusses some ways to achieve better performance from SQLite and shares some tricks. <ul> <li>transactions, concurrency, and autocommit</li> <li>user-defined functions</li> <li>using pragmas</li> <li>compilation flags</li> </ul></li> </ul> <p><strong>Michael #4: <a href="">The graphing calculator that makes learning math easier.</a></strong></p> <ul> <li>A full graphing calculator</li> <li>Programmable in Python</li> <li>Exam approved: Take the SAT and the ACT.</li> <li>Free browser emulator</li> </ul> <p><strong>Brian #5:</strong> <a href=""><strong>Installing Python Packages from a Jupyter Notebook</strong></a></p> <ul> <li>Jake VanderPlas</li> <li>using conda import sys !conda install --yes --prefix {sys.prefix} numpy</li> <li>using pip import sys {sys.executable} -m pip install numpy</li> <li>plus a discussion of why this is weird in Jupyter</li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>Videos from PyConDE 2017 are online</strong></a></p> <ul> <li>via Miroslav Šedivý <a href="">@eumiro</a></li> <li>Lots of interesting talk titles</li> <li>Almost all in English</li> </ul>
Dec 26, 2017
#57 Our take on Excel and Python
<p>Sponsored by DigitalOcean: <a href=""><strong></strong></a></p> <p><strong>Brian #1:</strong> <a href=""><strong>Testing Python 3 and 2 simultaneously with retox</strong></a></p> <ul> <li>Anthony Shaw</li> <li><a href="">tox</a> allows you to run the same tests in multiple configurations. <ul> <li>For example, multiple Python interpreters (2 vs 3), or on different hardware, or using different options, etc.</li> <li>tox can also tests your packaging code (on by default, but can be disabled)</li> </ul></li> <li><a href="">detox</a> allows multiple configurations to be tested in parallel with multiprocessing <ul> <li>typically running all tests 2-4 times faster</li> </ul></li> <li><a href="">retox</a> does this with a GUI <ul> <li>also adds “watch” capability</li> </ul></li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>Robo 3T / RoboMongo</strong></a></p> <ul> <li>MongoDB GUI with embedded shell</li> <li>CLI interaction</li> <li>GUI when you want it</li> <li>No. 34 repository on GitHub</li> </ul> <p><strong>Brian #3:</strong> <strong>regular expressions</strong></p> <ul> <li><a href="">Regular Expressions Practical Guide</a> <ul> <li>Python examples for some common expressions</li> <li>How to use the built in re package for email addresses, URLs, phone numbers</li> <li>substitution with <code>re.sub()</code></li> <li>splitting a string with <code>re.split()</code></li> <li>what some of the escape shortcuts mean, like <code>\w</code> for word, <code>\s</code> for whitespace, etc.</li> <li>iterating through matches with <code>re.finditer()</code></li> <li>Using compiled expressions</li> </ul></li> <li><a href="">Regular Expressions for Data Scientists</a> <ul> <li>another great intro, that also talks about:</li> <li><code></code></li> <li><code>re.findall()</code></li> <li>match groups</li> </ul></li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>MongoEngine</strong></a></p> <ul> <li>MongoEngine is a Document-Object Mapper (think ORM, but for document databases) for working with MongoDB from Python.</li> <li>Map classes to MongoDB (think SQLAlchemy but for document databases)</li> <li>Adds features lacking from MongoDB <ul> <li>Schema</li> <li>Required fields</li> <li>Constraints</li> <li>Relationships</li> </ul></li> </ul> <p><strong>Brian #5:</strong> <a href=""><strong>Introducing PrettyPrinter for Python</strong></a></p> <ul> <li>a powerful, syntax-highlighting, and declarative pretty printer for Python 3.6</li> <li>goals <ul> <li>Implement an algorithm that tries very hard to produce pretty output, even if it takes a bit more work.</li> <li>Implement a dead simple, declarative interface to writing your own pretty printers. Python developers rarely write <code>__repr__</code> methods because they're a pain; no one will definitely write pretty printing rules for user-defined types unless it's super simple.</li> <li>Implement syntax-highlighting that doesn't break on invalid Python syntax.</li> </ul></li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>Excel and Python</strong></a></p> <ul> <li>Replace VBA</li> <li>Python in Excel as the main scripting language</li> <li>They need feedback (fill out their survey, upvote the issue)</li> </ul> <p>Our news</p> <p>Michael: * Webcast: <strong><a href=""></a></strong></p>
Dec 21, 2017
#56 The pendulum of time swings beautifully in PyPI
<p>Sponsored by Rollbar! <a href=""></a></p> <p><strong>Brian #1:</strong> <a href=""><strong>Pendulum</strong></a> <strong>for datetimes</strong></p> <ul> <li><a href="">on github</a></li> <li>See also <ul> <li><a href="">arrow</a></li> <li><a href="">maya</a></li> <li><a href="">datetime</a>, and <a href="">Dealing with datetimes like a pro in Python</a></li> <li><a href="">dateutil</a></li> </ul></li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>Flask asynchronous background tasks with Celery and Redis</strong></a></p> <ul> <li>Easiest way to a significant scalability to your app: queuing</li> <li>What is Celery: Celery is an asynchronous task queue/job queue based on distributed message passing. It is focused on real-time operation, but supports scheduling as well</li> <li>How Celery works: <ul> <li>Celery client: This will be connect your Flask application to the Celery task</li> <li>Celery worker: A process that runs a background task</li> <li>Message broker: The Celery client communicates to the Celery worker through a message broker (redis in this case)</li> </ul></li> <li>All examples on Windows </li> </ul> <p><strong>Brian #3:</strong> <a href=""><strong>Building a Simple Web App With Bottle, SQLAlchemy, and the Twitter API</strong></a></p> <ul> <li>Guest article on RealPython, by <a href="">Bob Belderbos</a> of <a href="">PyBytes</a></li> <li>Fun full project start to finish using <a href="">Tweepy</a> to load tweets.</li> <li>Ends with a bottle app running on Heroku</li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>Python extension for VSCode updated, now brought to you by Microsoft</strong></a></p> <ul> <li>Don Jayamanne, creator of the Python extension for Visual Studio Code, joins Microsoft</li> <li>Full announcement: <strong><a href=""></a></strong></li> <li>Had Don on Talk Python back on <a href=""><strong>episode 101</strong></a>.</li> <li>What does Microsoft Python team publishing the extension mean?</li> <li>For all practical purposes the transition should be transparent to you. Additionally: <ul> <li>The extension will remain open source and free</li> <li>Development will continue to be on GitHub, under the existing license</li> <li>More dev resources means (generally) faster turnaround on bug fixes and new features</li> </ul></li> <li>Microsoft is hiring for Visual Studio Code / Python! They are hiring devs immediately to continue and expand work on our Python support for Visual Studio Code. If you are passionate about developer tools and productivity, this could be an ideal endeavor!</li> </ul> <p><strong>Brian #5:</strong> <a href=""><strong>A Comprehensive Guide To Web Design</strong></a></p> <ul> <li>Crash course in web design principles, not the mechanics</li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>Requestium</strong></a></p> <ul> <li>Integration layer between Requests and Selenium for automation of web actions.</li> <li>Merges the power of <a href="">Requests</a>, <a href="">Selenium</a>, and <a href="">Parsel</a> into a single integrated tool for automatizing web actions.</li> <li>The library was created for writing web automation scripts that are written using mostly Requests but that are able to seamlessly switch to Selenium for the JavaScript heavy parts of the website, while maintaining the session.</li> <li>Features <ul> <li>Enables switching between a Requests' Session and a Selenium webdriver while maintaining the current web session.</li> <li>Integrates Parsel's parser into the library, making xpath, css, and regex much cleaner to write.</li> <li>Improves Selenium's handling of dynamically loading elements.</li> <li>Makes cookie handling more flexible in Selenium.</li> <li>Makes clicking elements in Selenium more reliable.</li> <li>Supports Chrome and PhantomJS.</li> </ul></li> </ul> <p><strong>Our news</strong></p> <ul> <li><a href="">Test &amp; Code 33: Testing in Data Science with Kathrine Jarmul</a></li> <li>Thanks to the 9 folks to that have left an Amazon review for <a href="">Python Testing with pytest</a>.</li> </ul>
Dec 14, 2017
#55 Flask, Flask, Flask, 3x Flask
<p>Sponsored by DigitalOcean: <a href=""></a></p> <p><strong>Brian #1</strong> <a href=""><strong>The Flask Mega-Tutorial</strong></a><strong>, reborn</strong></p> <ul> <li>This very popular tutorial, written in 2012, has been rewritten.</li> <li><a href="">Miguel Grinberg</a> has rewritten it with the help of a kickstarter campaign.</li> <li><a href="">Part 1 of the tutorial</a> is up, and he’s releasing 1 part per week.</li> <li>Want it faster, you can get it all in an <a href="">eBook</a> right now.</li> <li>A video version is coming in January.</li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>Django 2.0 Released</strong></a></p> <ul> <li>This release starts Django’s use of a <a href="">loose form of semantic versioning</a></li> <li>Features <ul> <li>A <a href="">simplified URL routing syntax</a> that allows writing routes without regular expressions.</li> <li>A responsive, <a href="">mobile-friendly contrib.admin</a>.</li> <li><a href="">Window expressions</a> to allow adding an OVER clause to querysets.</li> </ul></li> <li>Python 3 only</li> <li>django.contrib.auth <ul> <li>The default iteration count for the PBKDF2 password hasher is increased from 36,000 to 100,000.</li> </ul></li> <li>Lots more changes</li> </ul> <p><strong>Brian #3:</strong> <a href=""><strong>The Big Ol' List of Rules</strong></a></p> <ul> <li><a href="">Flake8</a> is a popular code linter that combines pyflakes, pycodestyle, and mccabe. <ul> <li><a href="">pycodestyle</a> is the new pep8 to enforce <a href="">PEP8</a> suggestions. These are mostly style guide items, and not actual bugs.</li> <li><a href="">pyflakes</a> is more like a traditional linter in that it catches things that are probably oversight or bugs.</li> <li><a href="">mccabe</a> is harder to explain, but it generally tells you if your code might be too complicated, using <a href="">Cyclomatic Complexity</a>.</li> </ul></li> <li>Flake8 produces error codes if your code has problems <ul> <li>Ennn and Wnnn for pycodestyle errors and warnings</li> <li>Fnnn for pyflakes errors</li> <li>Cnnn for mccabe errors</li> </ul></li> <li>The <a href="">The Big Ol' List of Rules</a> is a very nice breakdown of every error, what it means, and has links to other documents where they are defined.</li> <li>Very nice work from <a href="">Grant McConnaughey</a></li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>requests-staticmock</strong></a></p> <ul> <li>via <a href="">Anthony Shaw</a></li> <li>The Session object allows you to persist certain parameters across requests. It also persists cookies across all requests made from the Session instance, and will use <code>urllib3</code>'s <a href="">connection pooling</a>. So if you're making several requests to the same host, the underlying TCP connection will be reused, which can result in a significant performance increase</li> <li>A Session object has all the methods of the main Requests API.</li> <li><a href=""><strong>requests-staticmock</strong></a> is a static HTTP mock interface for testing classes that leverage Python requests with <strong>no</strong> monkey patching!</li> </ul> <p><strong>Brian #5:</strong> <a href=""><strong>PEP 557 -- Data Classes</strong></a> <strong>have been approved</strong></p> <ul> <li>You can play with them now if you want, with the <a href="">3.7.0a3 developer build</a>.</li> <li>However, <a href="">3.7 isn’t scheduled for release until June 2018</a>.</li> </ul> <p>Very short Example lifted directly from PEP 557 doc.</p> <pre><code>@dataclass class C: a: int # 'a' has no default value b: int = 0 # assign a default value for 'b' </code></pre> <p>In this example, both a and b will be included in the added __init__ method, which will be defined as:</p> <pre><code>def __init__(self, a: int, b: int = 0): pass </code></pre> <ul> <li>Why not just use <a href="">attrs</a>? (Also lifted from the pep doc) <ul> <li>attrs moves faster than could be accommodated if it were moved in to the standard library.</li> <li>attrs supports additional features not being proposed here: validators, converters, metadata, etc. Data Classes makes a tradeoff to achieve simplicity by not implementing these features.</li> </ul></li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>Quart: 3x faster Flask</strong></a></p> <ul> <li>Python has evolved since <a href="">Flask</a> was first released around 8 years ago, particularly with the introduction of asyncio. </li> <li>Asyncio has allowed for the development of libraries such as <a href="">uvloop</a> and <a href="">asyncpg</a> that are reported (<a href="">here</a>, and <a href="">here</a>) to improve performance far beyond what was previously possible. </li> <li>Quart provides the easiest transition for Flask apps to use asyncio as it shares the Flask-API.</li> <li><strong>tl;dr:</strong> Upgrading <a href="">this</a> Flask-pyscopg2 app to a Quart-asyncpg app gives a performance speedup of 3x without requiring a major rewrite or adjustment of the code</li> <li>View methods become async / await methods</li> </ul> <p><strong>Our news</strong></p> <p>Michael:</p> <ul> <li>Pythonic staff of enlightnement <ul> <li>I have already encountered the Pythonic Staff of Enlightenment, see <a href="">the photo that Anthony tweeted of you guys brandishing it at PyCon US</a>.</li> <li>Now so can you: <a href=""></a></li> </ul></li> </ul>
Dec 07, 2017
#54 PyAnnotate your way to the future
<p>Sponsored by DigitalOcean. They just launched Spaces, get started today with a free 2 month trial of Spaces by going to <a href=""><strong></strong></a></p> <p><strong>Brian #1:</strong> <a href=""><strong>The PSF awarded $170,000 grant from Mozilla Open Source Program to improve sustainability of PyPI</strong></a></p> <ul> <li>Situation <ul> <li>The Python Packaging Index (PyPI) is the principal repository of software packages for the Python programming language.</li> <li>There are over 100 million Python packages are downloaded from PyPI every week.</li> <li>The Python community depends on PyPI for the ongoing functioning of the entire Python ecosystem.</li> <li>There are no paid staff at the PSF who work on PyPI, and there are only a handful of people who contribute regularly.</li> <li>This leads to a situation where we have to depend on volunteers to be on-call for outages and respond to critical security vulnerabilities in core Python Infrastructure.</li> </ul></li> <li>Next steps <ul> <li>The first milestone for Warehouse is redirecting portions of the production to Warehouse including traffic for the simple index and package downloads. At that milestone Warehouse will be the main entryway to Python packages for all but a small fraction of the interactions PyPI sees.</li> <li>The bulk of the work will be bringing Warehouse to feature parity with the administrative capabilities users need from the Package Index. We'll keep you posted as we figure out when you can expect that to be true.</li> </ul></li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>Dropbox releases PyAnnotate</strong></a></p> <ul> <li>Mypy is an experimental optional static type checker for Python that aims to combine the benefits of dynamic (or "duck") typing and static typing</li> <li>You can develop programs with dynamic typing and add static typing after your code has matured, or migrate existing Python code to static typing.</li> <li><a href="">mypy</a> is great, but it only works after you have added type annotations to your codebase.</li> <li>To easy the pain of adding type annotations to existing code, we’ve developed a tool, <a href="">PyAnnotate</a>, that observes what types are actually used at runtime, and inserts annotations into your source code based on those observations. We’ve now open-sourced the tool.</li> <li>run your code with a special <a href="">profiling hook</a> enabled. </li> <li>This observes all call arguments and return values and records the observed types in memory. At the end of a run the data is dumped to a file in JSON format. </li> <li>A separate command-line utility can then read this JSON file and use it to add inline annotations to your source code.</li> </ul> <p><strong>Brian #3:</strong> <a href=""><strong>pytest-annotate is now open-source!</strong></a></p> <h2>- Use pyannotate without a driver file:</h2> <pre><code> pip install pytest-annotate # to run code while collecting types pytest --annotate-output=./annotate.json # to see what will change with type hint comments pyannotate --type-info ./annotate.json &lt;path_to_code&gt; # to modify code pyannotate -w --type-info ./annotate.json &lt;path_to_code&gt; </code></pre> <p><strong>Michael #4:</strong> <a href=""><strong>Run Python script as systemd service</strong></a></p> <ul> <li>Great for making your own “services” on Linux</li> <li>Incredibly easy, just follow the gist linked above</li> </ul> <p><strong>Brian #5:</strong> <strong><a href="">pytest 3.3.0 released</a></strong></p> <ul> <li><a href="">changelog</a> includes 12 new features</li> <li>Most excited about: <ul> <li>Now pytest displays the total progress percentage while running tests. </li> <li>Now captures and displays output from the standard logging module.</li> </ul></li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>Why d = {} is faster than d = dict()</strong></a> </p> <ul> <li>It turns out that using <code>str()</code>, <code>list()</code>, <code>dict()</code> and <code>tuple()</code> for creating empty sequences isn't as fast as their shorthand counterparts (<code>''</code>, <code>[]</code>, <code>{}</code>, <code>()</code>).</li> <li>We can inspect what happens with the dis module…</li> </ul> <p><strong>Extras</strong></p> <ul> <li>F1 eSports now more exciting than real F1: <a href=""></a></li> </ul>
Nov 29, 2017
#53 Getting started with devpi and Git Virtual FS
<p>Sponsored by Rollbar! Get the bootstrap plan at <a href=""></a></p> <p><strong>Brian #1:</strong> <a href=""><strong>Exploring Line Lengths in Python Packages</strong></a></p> <ul> <li>Jake VanderPlas <a href=""><strong>@</strong></a><a href="">jakevdp</a></li> <li>PEP8 recommends 79 character line lengths.</li> <li>This article looks at line lenghts used in NumPy, SciPy, Pandas, ScikitLearn, Matplotlib, and AstroPy</li> <li>In the form of a Jupyter notebook so you can follow along with all the code and graphs.</li> <li>Fitting the graphs to distributions.</li> <li>Closing questions from Jake: <ul> <li>Where do other Python packages fit on the mode/spread graph?</li> <li>Has the coding style in these packages, reflected in line length, evolved over time?</li> <li>How do individual contributors behave? Do they tend to have similar habits across packages?</li> <li>What do these distributions look like for code written in other languages?</li> </ul></li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>NumPy: Plan for dropping Python 2.7 support</strong></a></p> <ul> <li>The Python core team plans to stop supporting Python 2 in 2020. </li> <li>We found that supporting Python 2 is an increasing burden on our limited resources; thus, we plan to eventually drop Python 2.</li> <li>Our current plan is as follows. <ul> <li>Until <strong>December 31, 2018</strong>, all NumPy releases will fully support both Python2 and Python3.</li> <li>Starting on <strong>January 1, 2019</strong>, any new feature releases will support only Python3.</li> </ul></li> </ul> <p><strong>Brian #3:</strong> <a href=""><strong>How to Learn Pandas</strong></a></p> <ul> <li>Ted Petrou <a href="">@TedPetrou</a></li> <li>Alternating between reading documentation and using it for small projects.</li> <li>Getting the most out of documentation</li> <li>Using Jupyter Notebook</li> <li>Using Kaggle kernels, which are datasets in the form of Jupyter notebooks.</li> <li>Creating your own kernels</li> <li>Try answering questions on SO to test your knowledge</li> <li>Set up some projects.</li> </ul> <p><strong>Michael #4:</strong> <a href="">Microsoft and GitHub team up to take Git virtual file system to macOS, Linux</a></p> <ul> <li>Watch the <a href="">10 min Microsoft presentation</a> to understand this quickly.</li> <li>Git doesn’t work that well for larger projects <ul> <li>Yes, it was built for Linux (640 MB) </li> <li>But there are larger projects <ul> <li>Visual Studio and related tools: 3,000 MB (5x)</li> <li>Windows: 270 GB (421x), 4,000 people committing per day.</li> </ul></li> <li>solution was to develop Git Virtual File System (GVFS)</li> </ul></li> <li>Before virtual: <ul> <li>12 hours to clone</li> <li>3 hours to checkout</li> <li>8 min for <code>git status</code></li> <li>30 min to commit</li> </ul></li> <li>After virtual: <ul> <li>90 sec clone</li> <li>30 sec checkout</li> <li>3 sec <code>git status</code></li> <li>8 sec commit</li> </ul></li> <li>Microsoft says that, so far, about half of its modifications have been accepted upstream, with upstream Git developers broadly approving of the approach the company has taken to improve the software's scaling.</li> <li>GitHub is interested in this for the paid, enterprise side</li> <li>Currently Windows only but Microsoft and GitHub are also working to bring similar capabilities to other platforms, with macOS coming first, and later Linux.</li> </ul> <p><strong>Brian #5:</strong> <a href=""><strong>Getting started with devpi</strong></a></p> <ul> <li>Stefan Scherfke <a href=""><strong>@</strong></a><a href="">sscherfke</a></li> <li>A walkthrough of setting up and using <a href="">devpi</a> , a local mirror/cache/local store PyPI server.</li> <li>Setting up the server</li> <li>User management</li> <li>Working with package indexes</li> <li>Uploading packages</li> <li>Using it to point your pip at</li> </ul> <p><strong>Michael #6:</strong> <a href="">Marketing-for-Engineers</a></p> <ul> <li>A curated list of marketing tools and resources to grow your product</li> <li>such as: <ul> <li>finding beta testers</li> <li>growing first user base</li> <li>advertising project without a budget</li> <li>scaling marketing activities for building constant revenue streams.</li> </ul></li> <li>What is the hardest part of creating a successful product / web app? (hint: it’s not programming or having the idea)</li> </ul> <p><strong>Extras</strong></p> <ul> <li>Black Friday Sale for <a href=""><strong>Python Testing with pytest</strong></a> &amp; all other pragmatic eBooks. <ul> <li>From 11/22 through Friday 12/1, all ebooks at <a href=""></a> are 40% off.</li> <li>That makes the pytest book about $14</li> <li>Use coupon code <strong>turkeysale2017</strong> to get the discount.</li> </ul></li> <li><a href=""><strong>PyCon Columbia</strong></a>, <a href=""><strong>@</strong></a><a href="">pyconcolombia</a>, is February 9, 10 and 11 - 2018 and tickets are available now. (Recommended by Oscar Arbelaez)</li> </ul>
Nov 22, 2017
#52 Call your APIs with uplink and test them in the tavern
<p>Sponsored by DigitalOcean. They just launched Spaces, get started today with a free 2 month trial of Spaces by going to <a href=""><strong></strong></a></p> <p><strong>Brian #1:</strong> <a href=""><strong>Restful API testing</strong></a> <a href=""><strong>with Tavern</strong></a></p> <p><strong>Michael #2:</strong> <a href=""><strong>Uplink</strong></a></p> <ul> <li>RESTful client API via decorators</li> <li>Create a class to represent the API</li> <li>Add methods with arguments, map to API calls.</li> <li>e.g. </li> </ul> <div class="codehilite"><pre><span></span><code> <span class="nd">@get</span><span class="p">(</span><span class="s2">&quot;/users/{username}&quot;</span><span class="p">)</span> <span class="k">def</span> <span class="nf">get_user</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">username</span><span class="p">):</span> <span class="sd">&quot;&quot;&quot;Get a single user.&quot;&quot;&quot;</span> </code></pre></div> <ul> <li>Uplink includes support for concurrent requests with asyncio (for Python 3.4+) </li> <li>Twisted (for all supported Python versions)</li> <li>Not production ready, but very exciting.</li> </ul> <p><strong>Brian #3:</strong> <a href=""><strong>Using json-schema for REST API endpoint tests</strong></a></p> <p><strong>Sponsor DIGITAL OCEAN SPACES</strong> Get started today with a free 2 month trial of Spaces by going to <a href=""><strong></strong></a></p> <p><strong>Michael #4:</strong> <a href=""><strong>Live coding to music!</strong></a></p> <ul> <li>via Ian Watt</li> <li>Talk at PyCon UK by Ryan Kirkbride called “Programming Music for Performance: Live coding with FoxDot”</li> </ul> <p><strong>Brian #5:</strong> <a href=""><strong>Weekly Python Chat</strong></a></p> <p><strong>Michael #6:</strong> <a href=""><strong>10 common beginner mistakes in Python</strong></a></p> <ul> <li>Via checkIO:</li> </ul> <ol> <li>Incorrect indentation, tabs and spaces</li> <li>Using a Mutable Value as a Default Value</li> <li>Write a lot of comments and docstrings</li> <li>Scoping</li> <li>Edge cases first (let’s go easy on the indents)</li> <li>Copying</li> <li>Creating count-by-one errors on loops (range is half closed)</li> <li>Wrong capitalization</li> <li>Using class variables incorrectly</li> </ol> <p><strong>Our news</strong></p> <p>Michael: </p> <ul> <li>Flash briefing?</li> <li>Firefox Quantum!</li> </ul>
Nov 16, 2017
#51 How to make your code 80 times faster
<p>This episode is brought to you by Datadog: <a href=""></a></p> <p><strong>Brian #1:</strong> <a href=""><strong>Exploring United States Policing Data with Python</strong></a></p> <ul> <li>How to take a publicly available data set and, using jupyter, ipython, matplotlib, numpy, pandas, and scipy: <ul> <li>ask questions of the data and get answers</li> <li>visualize results with plots</li> <li>fill in and/or remove blank data</li> </ul></li> <li>The example is interesting, and easy to follow. It doesn’t explain all the code, just shows it. You can go look that stuff up later.</li> <li>A few notes as I worked through half of the example: <ul> <li>pip install scipy</li> <li>step 1.7</li> <li>end of 2.3, don't need <code>.to_html().replace('\n','')</code></li> <li>Use shift-enter to run a cell</li> </ul></li> <li>I’m concerned with the validity of the results due to the dropping of rows with missing data. Some columns aren’t used for some questions, so for those purposes, the data shouldn’t be abandoned.</li> <li>Still a very nice tutorial.</li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>How to make your code 80 times faster</strong></a></p> <ul> <li>Often hear people who are happy because PyPy makes their code 2 times faster or so. Here is a short personal story which shows PyPy can go well beyond that.</li> <li>Evolutionary algorithms: the ambitious plan was to automatically evolve a logic which could control a (simulated) quadcopter</li> <li>To drive the quadcopter, a Creature has a run_step method which runs at each delta_t <ul> <li>inputs is a numpy array containing the desired setpoint and the current position on the Z axis;</li> <li>outputs is a numpy array containing the thrust to give to the motors. To start easy, all the 4 motors are constrained to have the same thrust</li> </ul></li> <li>run_step is called at 100Hz</li> <li>simply tried to run this code on CPython: ~6.15 seconds/generation</li> <li>Then tried with PyPy 5.9: Ouch! We are ~5.5x slower than CPython. This was kind of expected: numpy is based on cpyext, which is infamously slow</li> <li>first obvious step is to use numpypy instead of numpy: PyPy+numpy, and more than 2x faster than the original CPython</li> <li>So, let's try to do better. As usual, the first thing to do is to profile and see where we spend most of the time. </li> <li>we know that the matrix is small, and always of the same shape. So, let's unroll the loop manually</li> <li><strong>Tada</strong>: This is around 80 (eighty) times faster than the original CPython+numpy implementation, and around 35-40x faster than the naive PyPy+numpypy one</li> </ul> <p><strong>Brian #3:</strong> <a href=""><strong>Giving Open-Source Projects Life After a Developer's Death</strong></a></p> <p><strong>Michael #4:</strong> <a href=""><strong>Solar Powered Internet Connected Lawn Sprinkler Project</strong></a></p> <ul> <li>via listener suggestion / written: Lenin</li> <li>a little project with Adafruit’s Feather HUZZAH board and MicroPython</li> <li>Combines with Home Assistant</li> <li>Mostly based on AdaFruit, they have a detailed list of the hardware used.</li> <li>based on the <a href=""><strong>MQTT protocol</strong></a>, which is a Client-Server <em>Internet of Things</em> connectivity protocol, comes with micropython</li> <li>Nice references back to AdaFruit tutorials</li> <li>Talk Python #108: MicroPython and Open Source Hardware at Adafruit: <strong><a href=""></a></strong></li> </ul> <p><strong>Brian #5:</strong> <strong>Some New Python Books</strong></p> <ul> <li><a href="">Python Tricks: A Buffet of Awesome Python Features</a> <ul> <li>by Dan Bader</li> </ul></li> <li><a href="">Illustrated Guide to Python 3</a> <ul> <li>by Matt Harrison</li> </ul></li> <li>While we’re at it, there are some older Python books that could use some review love. If you’ve read these, please leave a review. It helps more than you may realize. <ul> <li><a href="">Python Testing with pytest</a>, by Brian Okken</li> <li><a href="">Test-Driven Development with Python</a>, by Harry Percival</li> <li><a href="">Two Scoops of Django</a>, Daniel &amp; Audrey Roy Greenfield</li> </ul></li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>Anaconda Distribution 5.0 released</strong></a></p> <ul> <li>Over 100 packages have been updated or added to the distribution. JupyterLab alpha preview 0.27.0 is now included, and MKL has been updated to 2018.0.0.</li> <li>The new version features all new compilers on macOS and Linux, providing substantial security and performance improvements.</li> <li>Where possible, all build recipes are now using conda-forge as a base, via</li> <li>A new channel, pkgs/main, has been added to defaults. The new channel is given top priority within defaults and holds packages built with the new compiler stack.</li> <li>The new version of Anaconda Distribution now features more flexible dependency pinning.</li> </ul>
Nov 09, 2017
#50 Bundling , shipping, and protecting Python applications
<h1>Python Bytes 50</h1> <p>Sponsored by DigitalOcean. They just launched Spaces, get started today with a free 2 month trial of Spaces by going to <a href=""><strong></strong></a></p> <p><strong>Brian #1:</strong> <a href=";list=PL85KuAjbN_gtuA4pNPftJWaui-8ARervQ&amp;index=2"><strong>Think Like a Pythonista</strong></a></p> <ul> <li>2017, by <a href="">@standupdev</a> Luciano Ramalho</li> <li><a href="">The PyBay2017 playlist</a></li> <li>Covered in “Think Lika a Pythonista” <ul> <li>Creating a container type, a Deck of Cards.</li> <li>Luciano shows how to utilize duck typing, builtin types, and operator overloading while creating a type without inheritance.</li> <li>Uses a <a href="">Jupyter notebook</a> to work with the code.</li> <li>Describes and shows monkeypatching to implement shuffle.</li> <li>Watch until the end, he takes feedback from the audience to optimize some code.</li> </ul></li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>Serpent.AI - Game Agent Framework</strong></a></p> <ul> <li>Turn ANY video game in a sandbox environment for AI &amp; Bot programming with Python.</li> <li>goal with Serpent.AI is to lower the barriers to entry when it comes to using games as sandboxes for code experiments. </li> <li>It unlocks your entire existing game library (Steam, DRM-Free etc.) to be used as potential game agent environments and it does so natively</li> <li>It also doesn't try to dictate how you solve your problems. Your game agent is your canvas!</li> <li>Even a twitch channel with live PyCharm + Jupyter + Game. Here’s a cool example: <a href=""></a></li> <li>Provides some useful conventions but is absolutely NOT opinionated about what you put in your agents: <ul> <li>Want to use the latest, cutting-edge deep reinforcement learning algorithm? ALLOWED. </li> <li>Want to use computer vision techniques, image processing and trigonometry? ALLOWED. </li> <li>Want to randomly press the Left or Right buttons? <em>sigh</em> ALLOWED. </li> </ul></li> </ul> <p><strong>Brian #3:</strong> <a href=""><strong>MkDocs</strong></a></p> <ul> <li>I’ve been creating pytest plugins using the <a href="">pytest-plugin cookiecutter</a>.</li> <li>One option is to start the documentation using mkdocs, so I thought I’d try it out.</li> <li>For the most part, you have a yaml file to configure things, and a directory with markdown files in it. Then you call <code>mkdocs build</code> and blammo, your documentation is built. I like markdown, so I’m going to try working with mkdocs more.</li> <li>Also want to try: <ul> <li>Generating documentation from source code using <a href="">Christian Medina</a>’s <a href="">How to write your own Python documentation generator</a> article and the code in a snippet, <a href=""></a>.</li> <li>I know about <a href="">Sphinx</a>, but I’m not a fan of reStructured text.</li> </ul></li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>PyInstaller 3.3 released</strong></a></p> <ul> <li>PyInstaller is a program that freezes (packages) Python programs into stand-alone executables, under Windows, Linux, Mac OS X, FreeBSD, Solaris and AIX.</li> <li>The main goal of PyInstaller is to be <strong>compatible with 3rd-party packages out-of-the-box</strong>.</li> <li><strong>Libraries like PyQt, Django or matplotlib are fully supported</strong>, without having to handle plugins or external data files manually. </li> <li>The #1 thing that stands out to me in this release: Python 3.6 support!</li> <li>PyInstaller can produce single immutable self contained dependency free portable exe files using the --one-file option.</li> <li>Consider the --noconsole option too</li> <li>cx_freeze vs pyinstaller? I can tell you that pyinstaller does a much better job of actually detecting and including dependencies. I recently tried both for freezing a multi-threaded, scipy based application and cx_freeze was a real hassle to get functional. Pyinstaller more or less just magically worked in my case whereas cx_freeze took hours of debugging.</li> </ul> <p><strong>Brian #5:</strong> <a href=""><strong>PEX:</strong></a> <a href=""><strong>A library and tool for generating .pex (Python EXecutable) files</strong></a></p> <ul> <li>Developed by twitter. Originally part of the twitter.commons package.</li> <li><code>pex</code> is a tool to create PEX files, which are “files are self-contained executable Python virtual environments.”, from <a href=""></a>.</li> <li>Python can import from zip files. You can add instructions at the beginning of a zip file to make it look like a python script. <code>pex</code> allows you to do that.</li> <li>Watch <a href="">WTF is PEX?</a>, a 16 min video describing how it works. Brian Wickman</li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>Using Cython to protect a Python codebase</strong></a></p> <ul> <li>A Python project that required the whole codebase to be protected</li> <li>They used Cython</li> <li>By following this guide, you should be able to cythonize a Python codebase with non-trivial package/module structure, thus making it difficult for evil hackers to reverse engineer it and steal your programming know-how.</li> <li>Although protecting Python sources from reverse engineering seems like a futile task at first, cythonizing all the code leads to a reasonable amount of security</li> <li>This was a Flask project!</li> <li>The current standard for Python archives is the wheel format (.whl), which aims to replace the .egg format. So, let's try to create a wheel with <code>python bdist_wheel</code>!</li> <li>Apparently, the archive contains not only compiled code, but also the sources.</li> <li>There is a way to fix this, however it is counter-intuitive.</li> </ul> <h2>Our news</h2> <ul> <li>Python for Windows developers: A survey → <a href=""></a>-</li> </ul>
Nov 02, 2017
#49 Your technical skills are obsolete: now what?
<p>Sponsored by DigitalOcean. They just launched Spaces, get started today with a free 2 month trial of Spaces by going to <a href=""><strong></strong></a></p> <p><strong>Brian #1: Conference videos for DjangoCon 2017 and PyGotham 2017</strong></p> <ul> <li><a href="">PyGotham 2017 videos on pyvideo</a></li> <li><a href=";list=PL2NFhrDSOxgXmA215-fo02djziShwLa6T">DjangoCon 2017 on YouTube</a></li> <li>One I’ve watched so far: <ul> <li><a href=";index=31&amp;list=PL2NFhrDSOxgXmA215-fo02djziShwLa6T">DjangoCon US 2017 - Django vs Flask by David &quot;DB&quot; Baumgold</a></li> <li><a href="">slides</a></li> <li>Very good introduction to Flask while comparing some of the features of Django to Flask and what the current frequent practices are for doing things in Flask like:</li> <li>Data modeling with SQLAlchemy, MongoEngine, or Peewee</li> <li>User admin with Flask-Security, which wraps Flask-Login, Flask-Permissions, and other commonly used together packages.</li> <li>Blueprints in Flask solve a similar problem as apps in Django.</li> <li>Flask-Marshmallow for APIs</li> </ul></li> </ul> <p><strong>Michael #2: <a href="">Python 3.6.3 released on Tue. All machines at FB are already running it</a> (3 days)</strong></p> <ul> <li>Tweet: Did you hear that 3.6.3 was released on Tue? How about that all machines at FB are already running it? Over 36.3% of our Python apps are 3.6 via <a href="">@llanga</a></li> <li>See Jason Fried’s presentation on culture: <a href="">Rules for Radicals: Changing the Culture of Python at Facebook</a></li> <li>More Python 3 news <ul> <li>Ubuntu 17.10: <em>“Python 2 is no longer installed by default. Python 3 has been updated to 3.6.”</em></li> <li>PSA: Python 3.3 is end-of-life in 2 days. Are you prepared?</li> </ul></li> </ul> <p><strong>Brian #3:</strong> <a href=""><strong>Your technical skills are obsolete: now what?</strong></a></p> <ul> <li>by Itamar Turner-Trauring</li> <li>We’re big proponents of keeping your skills current, of learning new techniques and technologies. But how does that fit in with life and work.</li> <li>This article is an opinion of how to work on new skills while at work, do it quickly, and look good to your manager.</li> <li>It starts with a good discussion of real business reasons why some projects use older technology. Basically, cost vs benefit of change.</li> <li>Steps to be part of the solution: <ul> <li>Identify obsolete and problematic technologies.</li> <li>Identify potential replacements.</li> <li>Get management buy in to get resources (you) to do a pilot project exploring new technology.</li> </ul></li> <li>This process will help you be better at identifying problems, even if you don’t get approval to fix it.</li> <li>He ends with a comment that if you don’t get approval, all is not lost, you have skills to apply to a new job.</li> <li>I’d like to make sue you do a few more steps before giving up and looking for a new job. Before you consider a move to a new team or company, I think… <ul> <li>You should give your manager the benefit of the doubt and use this to start a conversation. Make sure you understand their reasons for saying no.</li> <li>Make sure you are not proposing too much time taken away from your primary role in the company.</li> <li>State that you want to improve your skills by providing value for the team and the company. </li> <li>Is the “no” due to just bad timing? Is there a higher priority problem to work on? </li> <li>You’ve just shown that you are someone interested in keeping your skills sharp and helping the company by expanding your role. If you’re still stuck at this point, then consider a move but also, …</li> </ul></li> <li>Read this: <ul> <li><a href="">Team Geek: A Software Developer's Guide to Working Well with Others</a> - Brian Fitzpatrick</li> <li>Especially: - pg 117 : “Offensive vs Defensive work”. 50-70% of your time at work needs to be focused on creating new value for your company or your customers. No more than 30-50% on repaying technical debt. (Okken: Limit your process improvement / new technology exploration to no more than 10-20%, but try to never drop it below 5% of your time) - pg 113 : “It’s easier to ask for forgiveness than permission.” This is a fine line between doing the right thing and doing something you can get reprimanded for. Use good judgement. - Forgotten page number: A big part of your job is making your boss’s job easier and making your boss and your team look good.</li> </ul></li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>Visualizing Garbage Collection Algorithms</strong></a></p> <ul> <li>By Ken Fox</li> <li>Follow up from the excellent deep dive article in GC from Brian</li> <li>Most developers take automatic garbage collection for granted.</li> <li>It’s very difficult to see how GCs actually work.</li> <li><strong>GCs visualized</strong> (click on each image): <ul> <li><strong>Cleanup At The End: aka No GC</strong> (e.g. Apache web server creates a small pool of memory per request and throws the entire pool away when the request completes)</li> <li><strong>Reference Counting Collector</strong> (e.g. Python’s first pass GC, Microsoft COM, C++ Smart Pointers. Memory fragmentation is interesting) <ul> <li>The red flashes indicate reference counting activity. A very useful property of reference counting is that garbage is detected as soon as possible — you can sometimes see a flash of red immediately followed by the area turning black.</li> </ul></li> <li><strong>Mark-Sweep Collector</strong> (e.g. is this Python’s secondary collector? Probably is my guess) <ul> <li>Mark-sweep eliminates some of the problems of reference count. It can easily handle cyclic structures and it has lower overhead since it doesn’t need to maintain counts.</li> </ul></li> <li><strong>Mark-Compact Collector</strong> (Oracle’s Hotspot JVM’s tenured object space uses mark-compact) <ul> <li>Mark-compact disposes of memory, not by just marking it free, but by moving objects down into the free space</li> <li>The crazy idea of moving objects means that new objects can always just be created at the end of used memory. This is called a “bump” allocator and is as cheap as stack allocation, but without the limitations of stack size.</li> </ul></li> <li><strong>Copying Collector, aka Generational GC</strong> <ul> <li>The foundation of most high-performance garbage collection systems</li> </ul></li> </ul></li> </ul> <p><strong>Brian #5:</strong> <a href=""><strong>pathlib — Filesystem Paths as Objects</strong></a></p> <ul> <li>from Doug Hellman’s PyMOTW-3</li> <li>pathlib was introduced with Python 3.4</li> <li>If you need to work with the file system in Python, you should be using pathlib.</li> <li>Doug’s article is a really good overview.</li> <li>Features <ul> <li>Building paths with overloaded / operator</li> <li>Parsing paths with <code>.parts</code>, <code>.parents</code>, <code>.suffix</code>, <code>.stem</code></li> <li>Concrete paths such as <code>Path.home()</code>, <code>Path.cwd()</code></li> <li>Getting directory contents with <code>.iterdir()</code></li> <li>Pattern matching with <code>.glob()</code> and <code>.rglob()</code></li> <li>Reading and writing files with path objects.</li> <li>Working with directories and symbolic links</li> <li>File properties, permissions</li> <li>Deleting files and directories</li> </ul></li> <li>see also <ul> <li><a href=""></a></li> <li>The official docs are pretty good too</li> </ul></li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>LUMINOTH: Open source Computer Vision toolkit</strong></a></p> <ul> <li>Deep Learning toolkit for Computer Vision</li> <li>Supports object detection and image classification, but are aiming for much more.</li> <li>It is built in Python, using <a href="">TensorFlow</a> and <a href="">Sonnet</a> (Google’s Deep Learning framework and DeepMind’s graph library)</li> <li>Easily train neural networks to detect and classify objects with custom data. </li> <li>Use state of the art models such as Faster R-CNN (Region-based Convolutional Neural Networks)</li> <li>Comes with GPGPU support</li> <li>Simple training <ul> <li>Train your model by just typing lumi train. Do it locally or using Luminoth's built-in Google Cloud Platform support to train in the cloud.</li> <li>Once training is done, you can use our Tensorboard integration to visualize progress and intermediate results. </li> </ul></li> <li>Are also working on providing <strong>pre-trained checkpoints</strong> on popular datasets such as <a href="">Pascal VOC2012</a></li> </ul> <p><strong>Bonus article:</strong></p> <p><a href="">The Cleaning Hand of Pytest - My experiences with different approaches to testing</a></p> <ul> <li>by Wiktor Żurawik</li> <li>Two case studies of having to use unittest after using pytest</li> <li>Be sure to check out the “useful links” at the end of the article.</li> </ul> <h2>Our news</h2> <ul> <li><a href="">PyTennessee 2018</a></li> <li>Movie: All work, all play (available on netflix, here’s the <a href="">trailer</a>)</li> </ul>
Oct 25, 2017
#48 Garbage collection and memory management in Python
<p>Sponsored by DigitalOcean. They just launched Spaces, get started today with a free 2 month trial of Spaces by going to <a href=""><strong></strong></a></p> <p><strong>Brian #1:</strong> <a href=""><strong>The Python Graph Gallery</strong></a></p> <ul> <li>“cool graphs” x “head explodes with options”</li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>pynesis</strong></a></p> <ul> <li>High level python library for using kinesis streams</li> <li><p>What are Kinesis streams? <a href="http://Amazon Kinesis Streams">AWS Kinesis streams</a></p> <ul> <li>Enables you to build custom applications that process or analyze <a href="">streaming data</a> for specialized needs.</li> <li>Continuously capture and store terabytes of data per hour from hundreds of thousands of sources such as website clickstreams, financial transactions, social media feeds, IT logs, and location-tracking events.</li> </ul></li> <li><p>High level kinesis client. Support python 2.7 and 3.6, and has helpers for using it within Django.</p></li> <li>Some features: <ul> <li>Supports python 2 &amp; 3</li> <li>Django helpers included</li> <li>Automatically detects shard count changes</li> <li>Checkpoints/sequences persistence can be customized</li> <li>Provided Checkpointer implementations for memory, django model and redis</li> <li>Provided Dummy kinesis implementation for development/testing</li> </ul></li> </ul> <p><strong>Brian #3:</strong> <a href=""><strong>Things you need to know about garbage collection in Python</strong></a></p> <p><strong>Michael #4: WSGI Is Not Enough Anymore,</strong> <a href=""><strong>part 1</strong></a> <strong>and</strong> <a href=""><strong>part 2</strong></a></p> <ul> <li>Explores the factors that make WSGI a less attractive option for developing web applications with Python.</li> <li>Most major web frameworks use WSGI (Pyramid, Flask, Django, Bottle, etc.)</li> <li>This has been a major benefit / breakthrough</li> <li>The Web Server Gateway Interface (WSGI) is a <a href="">specification</a> which was first developed in 2003, and then revised in 2010, in order to create a standard for Python web frameworks to interact with web servers.</li> <li>The bad news is that WSGI comes with two major drawbacks: <ul> <li>WSGI compatible servers are synchronous</li> <li>WSGI compatible servers only supports the HTTP protocol</li> </ul></li> <li>Problem 1: Concurrency <ul> <li>By design, a WSGI server is synchronous. This means it blocks each request until a response arrives from the application.</li> <li>Scaling is done necessarily via threads (with GIL limitations), web gardens (multiple processes per server), and web farms (multiple servers)</li> </ul></li> <li>Problem 2: HTTP is the only protocol <ul> <li>HTML5 introduced, among other things, web sockets, which create a bi-directional communication layer between servers and clients.</li> <li>But they are not supported, so polling (yuck) is the only option</li> <li>Python frameworks which rely on WSGI do not implement web socket communication and must rely on 3rd party solutions and extra components and resources.</li> </ul></li> <li>Part 2 discusses solutions via event driven programming</li> <li>Part 3 (pending) talks about libraries for solving the concurrent problem in Python</li> </ul> <p><strong>Brian #5:</strong> <a href=""><strong>Queues in Python</strong></a></p> <ul> <li>Dan Bader</li> <li>I was in search of a LIFO queue and ran across this article by Dan.</li> <li>For LIFO:</li> </ul> <pre><code> ### collections.deque as LIFO queue q = collections.deque() # insert elements q.appendleft(item) #iterate for item in q: print(item) ### queue.LifoQueue q = queue.LifoQueue() # insert elements q.put(item) #iterate while not q.empty(): item = q.get() print(item) ### list as LIFO queue q = [] # insert elements q.append(item) #iterate for item in q[::-1]: print(item) </code></pre> <p><strong>Michael #6:</strong> <a href=""><strong>Using Reflection: A Podcast About Humans Engineering</strong></a></p> <ul> <li>by Mark Weiss</li> <li>Check out Jesse Davis’s <a href="">episode for a starter</a>.</li> <li>Engineering journey, what they value about engineering and skills they have come to recognize in themselves.</li> <li>Dig into what makes teams successful, and how we help them succeed.</li> </ul> <h2>Our news</h2> <ul> <li>Michael: Free MongoDB course has had over 5,000 signups in the first few days. Check it out <strong><a href=""></a></strong></li> </ul>
Oct 19, 2017
#47 PyPy now works with way more C-extensions and parking your package safely
<p>Sponsored by DigitalOcean. They just launched Spaces, get started today with a free 2 month trial of Spaces by going to <a href=""><strong></strong></a></p> <p><strong>Brian #1</strong>: <a href=""><strong>PyPy v5.9 Released, Now Supports Pandas, NumPy</strong></a></p> <ul> <li>NumPy and Pandas work on PyPy2.7 v5.9</li> <li>Cython 0.27.1 (released very recently) supports more projects with PyPy, both on PyPy2.7 and PyPy3.5 beta</li> <li>Optimized JSON parser for both memory and speed.</li> <li>CFFI updated</li> <li>Nice to see continued improvements and work on PyPy</li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>WTF Python?</strong></a></p> <ul> <li>Python, being awesome by design high-level and interpreter-based programming language, provides us with many features for the programmer's comfort. </li> <li>But sometimes, the outcomes of a Python snippet may not seem obvious to a regular user at first sight.</li> <li>Here is a fun project attempting to collect such classic and tricky examples of unexpected behaviors in Python and discuss what exactly is happening under the hood!</li> <li>Examples: <ul> <li><a href=""><strong>Skipping lines?</strong></a></li> <li><strong>​​</strong><a href=""><strong>Modifying a dictionary while iterating over it</strong></a></li> <li><a href=""><strong>Backslashes at the end of string</strong></a></li> <li><a href=""><strong>is is not what it is!</strong></a></li> </ul></li> <li>I’m thinking of doing some fun follow on projects with this. More on that later.</li> </ul> <p><strong>Brian #3:</strong> <a href=""><strong>Python Exercises</strong></a></p> <ul> <li>“… focus on the language itself and the standard library.”</li> <li>Some non-obvious Python exercises to help hone your Python skills, and possibly use in coding exercises of a job interview or maybe pre-interview screen.</li> <li>Topics <ul> <li>Basic syntax</li> <li>Text Processing</li> <li>OS Integration</li> <li>Functions</li> <li>Decorators &amp; Generators</li> <li>Classes, Modules, </li> <li>Exceptions, Lists, Dictionaries, Multiprocessing</li> <li>&amp; Testing! always including testing when ~~interviewing someone~~ practicing your coding.</li> </ul></li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>Exploiting misuse of Python's &quot;pickle&quot;</strong></a></p> <ul> <li>If you program in Python, you’re probably familiar with the pickle serialization library, which provides for efficient binary serialization and loading of Python datatypes.</li> <li>Hopefully, you’re also familiar with the warning printed prominently near the start of pickle’s documentation:</li> </ul> <p><em>Warning: The pickle module is not intended to be secure against erroneous or maliciously constructed data. Never unpickle data received from an untrusted or unauthenticated source.</em></p> <ul> <li>this blog post will describe exactly how trivial it is to exploit such a service, using a simplified version of the code I recently encountered as an example. </li> <li>Executing Code: So, what can we do with a vulnerable service? Well, pickle is supposed to allow us to represent arbitrary objects. An obvious target is Python’s <code>subprocess.Popen</code> objects!</li> </ul> <p><strong>Brian #5:</strong> <a href=""><strong>A Complete Beginner's Guide to Django</strong></a></p> <ul> <li>Lots of Django tutorials already, but this may appeal to folks with a more academic bent.</li> <li>Complete with wireframes, UML class hierarchies and use case diagrams.</li> <li>Series with 6 parts done, a 7th part planned, which will be the last part.</li> <li>Some fun comic like drawings, and lots of screenshots.</li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>pypi-parker</strong></a></p> <ul> <li>Helper tooling for parking PyPI namespaces to combat typosquatting.</li> <li>pypi-parker lets you easily park package names on PyPI to protect users of your packages from typosquatting.</li> <li>Typosquatting is a problem: in general, but also on PyPI. </li> <li>There are efforts being taken by pypa to protect core library names, but this does not (and really cannot and probably should not attempt to) help individual package owners.</li> <li>For example, <code>reqeusts</code> rather than <code>requests</code>, or <code>crytpography</code> rather than <code>cryptography</code>.</li> <li>Why? Self-serve is a good thing. Let's not try and get rid of that. Work with it instead.</li> <li>What? pypi-parker provides a custom distutils command park that interprets a provided config file to generate empty Python package source distributables. These packages will always throw an ImportError when someone tries to install them. You can customize the ImportError message to help guide users to the correct package.</li> </ul> <p>Our news</p> <p>Michael: </p> <ul> <li>Just launched <a href=""><strong></strong></a> Come and sign up to learn MongoDB and some Python</li> <li><a href=";cn=ZmxleGlibGVfcmVjc18y&amp;refsrc=email&amp;iid=94524f9448ff4f01905c15700bf2cc1c&amp;uid=3098427092&amp;nid=244+272699400"><strong>Python3 usage has doubled in the past year</strong></a> (thanks Donald Stufft)</li> </ul>
Oct 12, 2017
#46 Spicy lecture notes and unicorn console spinners
<p>Sponsored by DigitalOcean. They just launched Spaces, get started today with a free 2 month trial of Spaces by going to <a href=""><strong></strong></a></p> <p><strong>Brian #1:</strong> <a href=""><strong>Scipy lecture notes</strong></a></p> <ul> <li>“One document to learn numerics, science, and data with Python”</li> <li>Topics <ul> <li>Python language tutorial</li> <li>NumPy, Matplotlib, scipy</li> <li>Debugging, optimizing, image manipulation</li> <li>Statistics, scikit-image, scikit learn</li> <li>3D plotting</li> </ul></li> <li>Nice table of contents layout that makes it easy to jump right to whatever you need to learn.</li> <li>Just in time learning for scientific Python.</li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>Building a desktop notification tool for Linux using python</strong></a></p> <ul> <li>The term desktop notifications refer to a graphical control element that communicates certain events to the user without forcing them to react to this notification immediately.</li> <li>Example: we are going to build a notification tool which displays the current rate of bitcoins in INR.</li> <li>Based on <a href="">notify2</a> package</li> </ul> <p><strong>Brian #3<em>*</strong>:</em>* <a href=""><strong>pytest-benchmark</strong></a></p> <ul> <li>Easily wrap some time constraints around some code to make sure certain parts of your system don’t slow down.</li> <li>Good table or graph based preliminary times with statistics</li> <li>Can generate golden sets of numbers, then compare against those and fail based on changes in particular stats like min, mean, etc.</li> <li>Can have max and min times for benchmarks even without previous training.</li> <li>Lots of fun flags and utilities.</li> <li>good integration with pytest</li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>Alice in Python projectland</strong></a></p> <ul> <li>via Vicki Boykis</li> <li>Python project structure and packaging standardization is still not a solved problem</li> <li>In the JVM, as long as you have your path structured correctly, build tools will understand it and create a package for you into an executable JAR.</li> <li>But, when I started looking for the same standardization in Python, it wasn’t as straightforward. Some questions I had as I worked: <ul> <li>Should I be using virtualenvs? </li> <li>Pipenvs? </li> <li>Setuptools? </li> <li>Should I have a setup.cfg? </li> <li>What are wheels, or eggs, for that matter? </li> <li>Does each folder need an </li> <li>What does that file even do? </li> <li>How do I reference modules along the same PYTHONPATH?</li> </ul></li> <li>Hat tip to <a href="">pipreqs</a></li> <li>Conclusion: Python project structure and packaging can be intimidating, but, if you take it step by step, it doesn’t have to be.</li> </ul> <p><strong>Brian #5:</strong> <a href=""><strong>How to teach technical concepts with cartoons</strong></a></p> <ul> <li>Just draw more pictures.</li> <li>You don’t have to be a good artist for drawings to help with retention when you are trying to teach technical concepts.</li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>Halo: Beautiful terminal spinners in Python</strong></a></p> <ul> <li>We’ve talk about progressbars: tqdm:</li> <li>doesn’t have to be.</li> <li>Cool methods like</li> <li><code>spinner.start([text])</code></li> <li><code>spinner.succeed([text])</code></li> <li><code>[text])</code></li> <li>Windows File Progress Dialog Author:</li> </ul> <h2>Extras</h2> <ul> <li>releases: stay current. go upgrade <ul> <li><a href="">Python 3.6.3 released</a></li> <li><a href="">pytest 3.2.3 released</a></li> </ul></li> <li>New Test &amp; Code episodes <ul> <li><a href="">31: I'm so sick of the testing pyramid</a></li> <li><a href="">32: David Hussman - Agile vs Agility, Dude's Law, and more</a></li> </ul></li> </ul>
Oct 05, 2017
#45 A really small web API and OS-level machine learning
<p>This episode is brought to you by Rollbar: <a href=""></a></p> <p><strong>Brian #1:</strong> <a href=""><strong>pico</strong></a> </p> <ul> <li>"a very small web application framework for Python" </li> <li>Recommended by Ivan Pejić</li> <li><a href=";;t=5h54m44s">lightning talk from EuroPython 2017</a></li> <li>This would be a good web framework for building internal services and tools that non-web developers need to interact with and modify.</li> <li>Very simple.</li> <li>Not REST, but not confusing either.</li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>High Sierra ships, first major OS with machine learning built in?</strong></a></p> <ul> <li>September 26th <a href=""><strong>macOS High Sierra</strong></a> was released (yay)</li> <li>Mostly a foundational release with barely visible changes but awesome things like APFS replacing HFS+, etc.</li> <li>Comes with <strong>CoreML</strong> <ul> <li>Apple’s intent with the new CoreML framework is to package up prebuilt ML models and execution engines and make them possible for third-party apps to use. </li> <li>Developers can take a trained machine learning algorithm, package it up as an MLModel, and integrate it into their apps. </li> <li>Apple offers a few default machine learning models that developers can download and use too</li> </ul></li> <li>Rather than sharing your data with a central server, grouping it together with a lot of other people's data, and improving machine learning models that way, Apple stresses that everything CoreML does is happening on the device.</li> <li>On Macs that support Metal—generally, Macs from 2012 and later—CoreML uses a mix of CPU processing and GPGPU processing, depending on the task.</li> <li>Add on the fact that High Sierra has <a href=""><strong>external GPU support</strong></a> now and you have a sweet combo.</li> </ul> <p><strong>Brian #3:</strong> <a href=""><strong>A guide to logging in Python</strong></a></p> <ul> <li>Simply put, the best logging introduction I've read so far.</li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>Let me introduce: slots</strong></a></p> <ul> <li>So what are slots? __slots__ are a different way to define the attributes storage for classes in Python.</li> <li>for normal Python classes, a dict is used to store the instance's attributes.</li> <li>With <code>__slots__</code> we don't have an attribute called <code>__dict__</code> inside our instance. But we have a new attribute called <code>__slots__</code>.</li> <li>But why would you need to use slots when you have a dict? Well the answer is that __slots__ are a lot lighter and slightly faster.</li> <li>Outcome: <ul> <li>~57% less memory usage thanks to just one line of code.</li> <li>__slots__ are also slightly faster.</li> </ul></li> <li>Covered in depth in my <a href="">Write Pythonic Code Like a Seasoned Developer</a> course.</li> </ul> <p><strong>Brian #5:</strong> <a href=""><strong>pipenv revisited</strong></a></p> <ul> <li>Covered in <a href="">episode 11</a>. However, there are some notable changes since then.</li> <li>Reminder: <ul> <li><code>pepenv</code> handles virtualenv and pip interaction for you</li> <li><code>pipenv install</code> creates a virtualenv (if one doesn't exist) and installs stuff into a virtualenv.</li> <li><code>pipenv shell</code> uses the virtualenv</li> <li><code>exit</code> allows you to get out of the virtualenv</li> <li><code>pipenv lock -r</code> will generate a requirements.txt file for you, so you can use it even if you need a requirements.txt file.</li> </ul></li> <li>Notable changes: <ul> <li>New <a href="">4 minute screencast</a> with Kenneth demonstrating how to use it. Watching him use it makes it very simple to understand. </li> <li>Specify multiple package indexes, and even specify a particular index for particular packages. So you can combine both pypi with a company index and a group index and maybe one for your project.</li> <li>pipenv check will tell you about any known security vulnerabilities in your installed packages</li> <li>9 months old with 192 releases, so keep an eye on it for new features all the time.</li> </ul></li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>Stack Overflow gives an even closer look at developer salaries</strong></a></p> <ul> <li>Tabs and spaces aren't the only things that influence developer pay…</li> <li>Some of the broad trends are no big surprise; for example, the chosen cities tend to pay more than their respective nations do, for example.</li> <li>DevOps specialists and data scientists both earn well. </li> <li>Other aspects of the data are a little more surprising. Graphics programmers, for example, aren't particularly well paid, in spite of having a relatively specialized, complex niche.</li> <li>And while earnings in four of the countries are surprisingly similar, those in America are substantially higher, regardless of experience; in fact, the median salary of a developer in the US is comparable to that of someone with 20 years of experience in Canada or Germany and markedly higher than 20-year veterans in France and the UK. Even after taking into account the US' higher healthcare costs, America is the place to be if you're a programmer.</li> <li>Comments <ul> <li>I do have to wonder how much Silicon Valley skews that salary chart, as the Web 2.0 companies pay HUGE comparatively [<a href=";post=34009615">ref</a>]</li> <li>I asked Stack Overflow's data scientist that question, and she said not much; even without its outlier cities, the US pays much more than the rest of the world. [<a href=";post=34009667">ref</a>]</li> <li>Healthcare cost are only part of it. I got paid about $600/month 9 months a year by my government to study in university. [<a href=";post=34009803">ref</a>]</li> <li>I feel like a lot of IT people lack soft skills, and it caps their salary at a lower end. [<a href=";post=34009925">ref</a>]</li> </ul></li> </ul> <p>Our news:</p> <ul> <li>Hardcopies of <a href="">Python Testing with pytest</a> now <a href="">shipping on Amazon</a>, as well as <a href="">Pragmatic</a>. <ul> <li>When you get your copy, let me know. Send a pic to <a href="">@brianokken</a></li> </ul></li> </ul>
Sep 29, 2017
#44 pip install malicious-code
<p>This episode is brought to you by Datadog: <a href=""><strong></strong></a></p> <p><strong>Michael #1:</strong> <a href=""><strong>Ten Malicious Libraries Found on PyPI</strong></a></p> <ul> <li>Code packages available in PyPI contained modified installation scripts.</li> <li>Vulnerabilities were introduced into the execution of packages for approximately 20 packages on PyPI</li> <li>Package names that closely resembled those used for packages found in the standard Python library (e.g. <code>urlib</code> vs <code>urllib</code>)</li> <li>The packages contained the exact same code as the upstream libraries except for an installation script.</li> <li>Officials with the Slovak authority said they recently notified PyPI administrators of the activity, and all identified packages were taken down immediately. Removal of the infected libraries, however, does nothing to purge them from servers that installed them.</li> <li>From PSF: <em>Unlike some language package management systems, PyPI does not have any full time staff devoted to it. It is a volunteer run project with only two active administrators. As such, it doesn't currently have resources for some of the proposed solutions such as actively monitoring or approving every new project published to PyPI. Historically and by necessity we've relied on a reactive strategy of taking down potentially malicious projects as we've become aware of them.</em></li> <li>Comments <ul> <li><a href=";post=33997861">pip gets more paranoid in the install process</a></li> <li><a href=";post=34000031">Downloads were not super bad</a></li> <li><a href=";post=33999957">Stestagg is sitting on lots of misspellings</a> -<a href=";post=33999819">Undergrad thesis compromised Ruby and NodeJS too</a></li> </ul></li> <li>related: <ul> <li>original warning: <a href=""></a></li> <li>stdlib names no longer allowed: <a href=""></a></li> </ul></li> </ul> <p><strong>Brian #2: PyPI migration to Warehouse is in progress</strong></p> <ul> <li>Thanks to Jonas Neubert for researching this topic and writing a blog post titled <a href="">Publishing your First PyPI Package by/for the Absolute Beginner</a></li> <li>The steps to publish to PyPI have changed with the move to warehouse and</li> <li><a href=""></a> is no longer in read-only mode, it is where you publish packages</li> <li>The old APIs at <a href=""></a> are disabled, if you have a .pypirc file you'll have to update the URLs</li> <li>You no longer need to register package names before first uploading, the project gets created on the fly during the first upload of the package.</li> <li>The best way to update anything in a package is to change your local package and upload it again, see <a href=""></a>. <ul> <li>This includes even just changes to the description.</li> <li>Manual file upload is gone.</li> </ul></li> <li>As of right now it looks like you still need to register through, then do the rest of the interactions with See <a href=""></a></li> <li>Markdown support for package descriptions, like seems to be coming: <a href=""></a></li> <li>Jonas’ <a href="">blog post</a> is from 13 Sep 2017, so it might be the most up to date tutorial on all the steps to get a package onto PyPI.</li> </ul> <p><strong>Brian #3:</strong> <strong>Live coding in a presentation</strong></p> <ul> <li>Last week’s discussion of <a href="">David Beazley’s Fun of Reinvention talk</a> got me thinking about doing live coding during a presentation since he did it so well.</li> <li>Several links regarding how to do various levels of live coding: <ul> <li>Advice for live coding: <a href=""></a></li> <li>Not quite live coding: <a href=""></a></li> <li>Avoiding live coding: <a href=""></a></li> </ul></li> <li>Live coding: <ul> <li>practice, have a backup plan, don’t forget to talk, plan content</li> </ul></li> <li>not quite: <ul> <li>use git tags</li> </ul></li> <li>avoiding it: <ul> <li>My favorite effect is fade-in slideshows where part of the code is shown at a time so you can talk about it and people know which bit to look at</li> </ul></li> </ul> <p><strong>Michael #4: Notable REST / Web Frameworks</strong></p> <ul> <li><p><strong>Falcon: <a href=""></a></strong></p> <ul> <li>Unburdening APIs for over 4.70 x 10-2 centuries. (4.7 years)</li> <li>Falcon is a bare-metal Python web API framework for building very fast app backends and microservices.</li> <li><strong>Complementary:</strong> Falcon complements more general Python web frameworks by providing bare-metal performance and flexibility wherever you need it.</li> <li><strong>Compatible</strong>: Thanks to WSGI, Falcon runs on a large variety of web servers and platforms. Falcon works great with CPython 2.6, 2.7, and 3.3+. Try PyPy for an extra speed boost.</li> </ul></li> <li><p><strong>Hug: <a href=""></a></strong></p> <ul> <li>Drastically simplify API development over multiple interfaces. </li> <li>With hug, design and develop your API once, then expose it however your clients need to consume it. Be it locally, over HTTP, or through the command line.</li> <li>Built-in documentation</li> </ul></li> </ul> <p><strong>Brian #5:</strong> <strong>tox</strong></p> <ul> <li>“The name of the <a href="">tox automation project</a> derives from "testing out of the box". It aims to "automate and standardize testing in Python". Conceptually it is one level above pytest and serves as a command line frontend for running tests and automate all kinds of tasks around the project. It also acts as a frontend for <a href="">Continuous Integration Systems</a> to unify what you do locally and what happens in e.g. Jenkins or Travis CI.” - Oliver Bestwalter</li> <li>a small tox.ini file:</li> </ul> <pre><code> [tox] envlist = py27,py35, py36 [testenv] deps=pytest commands=pytest </code></pre> <ul> <li>You place this in your package source directory and then run tox, which will: <ul> <li>Use to create a sdist</li> <li>create a virtual environment for each environment in envlist</li> <li>Install dependencies in the environments</li> <li>Install your package into the environment</li> <li>Run the tests</li> <li>Do this for multiple environments, so multiple Python versions (as an example)</li> </ul></li> <li>Much more powerful than that, but that’s how many people use it.</li> <li>Further Reading: <ul> <li><a href=""></a></li> <li><a href=""></a> </li> <li><a href=""></a></li> </ul></li> </ul> <p><strong>Michael #6: flake8-tidy-imports</strong> <a href=""><strong>deprecated imports</strong></a></p> <ul> <li>You can declare {python2to3} as a banned-module import, and it will check against a long list of import moves and removals between python 2 and python 3, suggesting relevant replacements if available. </li> <li>I meticulously compiled this list by reading release notes from Python 3.0-3.6 as well as testing in a large legacy python codebase, but I presumably missed a few.</li> <li>Example:</li> </ul> <pre><code> flake8 I201 Banned import 'mock' used - use unittest.mock instead. </code></pre> <p><strong>Michael #7 (bonus!):</strong> <a href=""><strong>Help Me Offer Coaching to First-Time PyGotham Speakers</strong></a></p> <ul> <li>Via A. Jesse Jiru Davis</li> <li>I want to raise $1200 for public-speaking coaching for first-time speakers at PyGotham, the New York City Python conference. Will you chip in?</li> <li>Jesse is a PyGotham conference organizer, but I’m launching this fundraiser independently of PyGotham.</li> <li>As of September 19, I have raised my goal. Thanks to everyone who donated! <h2>Our news</h2></li> </ul> <p><strong>Michael</strong>: </p> <ul> <li>Finished writing my <strong>free MongoDB course</strong> (subscribe to get notified of release at <strong><a href=""></a></strong> )</li> <li><a href=""><strong>python-switch</strong></a> kind of went off the hook (see <a href="">this</a> and <a href="">that</a>)</li> </ul> <p><strong>Brian</strong>: </p> <ul> <li>Book is shipping: <a href="">Python Testing with pytest</a></li> </ul>
Sep 20, 2017
#43 Python string theory, v2
<h1>Python Bytes 43</h1> <p>This episode is brought to you by Rollbar: <a href=""><strong></strong></a></p> <p><strong>Brian #1:</strong> <a href=""><strong>future-fstrings</strong></a></p> <ul> <li>A backport of fstrings to python &lt; 3.6</li> <li>Include an encoding string the top of your file (this replaces the utf-8 line if you already have it)</li> <li>And then write python3.6 fstring code as usual!</li> </ul> <pre><code> # -*- coding: future_fstrings -*- thing = 'world' print(f'hello {thing}') </code></pre> <ul> <li>In action:</li> </ul> <pre><code> $ python2.7 hello world </code></pre> <ul> <li>I’m still undecided if I like this sort of monkeying with the language through the encoding mechanism back door. </li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>The Fun of Reinvention</strong></a></p> <ul> <li>Keynote from PyCon Israel</li> <li>David Beazley rocks it again</li> <li>Let’s take Python 3.6 features and see how far we can push them</li> <li>Builds an aspect-oriented constraint system using just 3.6 features</li> </ul> <p><strong>Brian #3:</strong> <a href=""><strong>Sound Pattern Recognition with Python</strong></a></p> <ul> <li>Using<code></code> to read a .wav file.</li> <li>Looking for peaks (knocks).</li> <li>Using minimum values to classify peaks, and minimum distance between peaks.</li> <li>This is an interesting start into audio measurements using Python.</li> <li>Would be fun to extend to some basic scope measurements, like sampling with a resolution bandwidth, trigger thresholds, pre-trigger time guards, etc.</li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>PEP 550: Execution Context</strong></a></p> <ul> <li>From the guys at <a href=""><strong></strong></a></li> <li>Adds a new generic mechanism of ensuring consistent access to non-local state in the context of out-of-order execution, such as in Python generators and coroutines.</li> <li>Thread-local storage, such as <code>threading.local()</code>, is inadequate for programs that execute concurrently in the same OS thread. This PEP proposes a solution to this problem.</li> <li>A few examples of where Thread-local storage (TLS) is commonly relied upon: <ul> <li>Context managers like decimal contexts,<code>numpy.errstate</code>, and <code>warnings.catch_warnings</code>.</li> <li>Request-related data, such as security tokens and request data in web applications, language context for<code>gettext</code> etc.</li> <li>Profiling, tracing, and logging in large code bases.</li> </ul></li> <li>The motivation from <a href=""><strong>uvloop</strong></a> is obviously at work here.</li> </ul> <p><strong>Brian #5:</strong> <a href=""><strong>Intro to Threads and Processes in Python</strong></a></p> <ul> <li>Beginner’s guide to parallel programming</li> <li>Threads and processes are both useful for different kinds of problems.</li> <li>This is a good quick explanation of when and where to use either. With pictures!</li> <li>Threads <ul> <li>Like mini processes that live inside one process.</li> <li>Share mem space with other threads.</li> <li>Cannot run simultaneously in Python (there are some workarounds), due to GIL.</li> <li>Good for tasks waiting on IO.</li> </ul></li> <li>Processes <ul> <li>Controlled by OS</li> <li>Can run simultaneously</li> <li>Good for CPU intensive work because you can use multiple cores.</li> </ul></li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>Alternative filesystems for Python</strong></a></p> <ul> <li>PyFilesystem: Filesystem Abstraction for Python. </li> <li>Work with files and directories in archives, memory, the cloud etc. as easily as your local drive.</li> <li>Uses <ul> <li>Write code now, decide later where the data will be stored</li> <li>unit test without writing real files</li> <li>upload files to the cloud without learning a new API</li> <li>sandbox your file writing code</li> </ul></li> <li>File system backends <ul> <li><a href="">AppFS</a> Filesystems for application data.</li> <li><a href="">S3FS</a> Amazon S3 Filesystem.</li> <li><a href="">FTPFS</a> File Transfer Protocol.</li> <li><a href="">MemoryFS</a> An in-memory filesystem.</li> <li><a href="">MountFS</a> A virtual filesystem that can <em>mount</em> other filesystems.</li> <li><a href="">MultiFS</a> A virtual filesystem that combines other filesystems.</li> <li><a href="">OSFS</a> OS Filesystem (hard-drive).</li> <li><a href="">TarFS</a> Read and write compressed Tar archives.</li> <li><a href="">TempFS</a> Contains temporary data.</li> <li><a href="">ZipFS</a> Read and write Zip files.</li> <li>and more</li> </ul></li> </ul> <h2>Our news</h2> <p>Michael: switch statement extension to Python: <a href=""><strong></strong></a></p>
Sep 14, 2017
#42 Behold: The Python 2 death clock
<p>Sponsored by DataDog! <a href=""></a></p> <p><strong>Brian #1:</strong> <strong>What Kenneth Did Last Week (well, recently)</strong></p> <ul> <li><p>Kenneth Reitz</p> <ul> <li><p>Homebrew Python Tap</p> <ul> <li>Python 2.5 through 3.6 available through homebrew </li> <li><a href=""></a></li> <li><pre><code>$ brew tap kennethreitz/pythons $ brew install python-2.5 </code></pre></li> </ul></li> <li><a href=""></a> <ul> <li>“ Twisted Deferred Thread backend for Requests.”</li> <li>Can be used with async/await or with twisted.</li> </ul></li> <li><a href=""></a> <ul> <li>“Runs things in the background.”</li> </ul></li> <li><a href=""></a> <ul> <li>“ (for humans)”</li> <li>“This repo exists to provide an example file, that can be used to bootstrap your next Python project. It includes some advanced patterns and best practices for, as well as some commented–out nice–to–haves.”</li> </ul></li> </ul></li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>Python 2 Death Clock</strong></a></p> <ul> <li>Python 2.7 will not be maintained past 2020. </li> <li>No official date has been given, so this clock counts down until April 12th, 2020, which will be roughly the time of the 2020 PyCon.</li> <li>I am hereby suggesting we make PyCon 2020 the official end-of-life date, and we throw a massive party to celebrate all that Python 2 has done for us.</li> <li>Python 2, thank you for your years of faithful service.</li> <li>Python 3, your time is now.</li> </ul> <p><strong>Brian #3:</strong> <a href=""><strong>Small Functions considered Harmful</strong></a></p> <ul> <li>Cindy Sridharan</li> <li>"General programming advice doled out invariably seems to extoll the elegance and efficacy of small functions."</li> <li>This is sometimes pushed to the extreme of having one line functions that are only called from one place. Understand that doing this increases your code size by 4 lines every time you do it. <ul> <li>1 line for the function call isn't removed because you moved the guts into a function.</li> <li>2 lines for function definition and guts</li> <li>2 lines to properly space your new function around other functions.</li> </ul></li> <li>Supposed Benefit: Do one thing; a function should only ever do one thing and do it well.</li> <li>Problems: <ul> <li>"Instead of a reasonably airtight abstraction that can be understood (and tested) as a single unit, we now end up with even smaller units that’ve been carved out to delineate each and every component of “the one thing” until it’s fully modular and entirely DRY." </li> <li>"...pragmatism and reason are sacrificed at the altar of a dogmatic adherence to DRY, ..."</li> <li>premature abstractions. breaking up the code into smaller functions before you really understand the problem space can make it harder to refactor later.</li> <li>micro-functions tend to have longer names because you need more names. Longer names aren't always a good thing when you have many long names on a page.</li> <li>loss of locality: One bit of functionality that used to be in one function is now spread across many functions and possibly moved far away from use.</li> <li>class pollution: class interfaces grow with smaller functions and hide the real intended interface.</li> <li>harder to read, especially for newcomers.</li> </ul></li> <li>There is still a place for small functions. But use it in moderation. Communicating with future developers clearly is more important than following dogmatic rules about function size.</li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>Why Python 3</strong></a></p> <ul> <li>All the cool Python 3 features that'll make you switch today!</li> <li>Presented as a random code sample surprise</li> <li>Examples: <ul> <li>Annotations: <code>def my_add(a: int, b: int) -&gt; int</code></li> <li>Keyword only arguments: <code>def f(a, b, *args, option=True)</code></li> <li>Yield from: <code>yield from range(5)</code></li> <li>Enums: <code>class Color(Enum)</code></li> </ul></li> </ul> <p><strong>Brian #5:</strong> <strong>EANABs</strong></p> <ul> <li>Equally Attractive Non-Alcoholic Beverage</li> <li>There is drinking that happens often when you get a bunch of adults together. Often with work or tech gatherings. That’s fine. But make sure you emphasize that drinking is not required. </li> <li><a href="">@treyhunner</a> <a href="">brought it up recently</a> and suggested that all conferences and tech events should have this. <ul> <li>"I sometimes feel excluded when events include nice alcohol but only cheap soda" <ul> <li><a href="">Stanford site</a> has a bunch of great recipes.</li> </ul></li> <li>“EANABS are required at all Stanford parties, …” <ul> <li>If you have specialty local beers, try to find specialty local sodas.</li> <li>If you have nice spiked punch, have a NA version also.</li> <li>If you have cocktails, advertise your ability to serve mocktails. </li> </ul></li> </ul></li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>The Incredible Growth of Python</strong></a></p> <ul> <li>via StackOverflow</li> <li>Recently explored how wealthy countries (those defined as high-income by the World Bank) tend to visit a different set of technologies than the rest of the world. </li> <li>Largest differences we saw was in the programming language Python. </li> <li>High-income countries, the growth of Python is even larger than it might appear from tools like Stack Overflow Trends, or in other rankings.</li> <li>[StackOverflow] makes the case that Python has a solid claim to being the fastest-growing major programming language.</li> <li>June 2017 was the first month that Python was the most visited tag on Stack Overflow within high-income nations. (Grown has grown by 2.5-fold since 2012)</li> <li><strong>Python compared to smaller, growing technologies</strong> graph is incredible.</li> <li>Also: <a href=""><strong>Python overtakes R, becomes the leader in Data Science, Machine Learning platforms</strong></a></li> </ul>
Sep 08, 2017
#41 Python Concurrency From the Ground Up and Caring for our Community
<p>Brought to you by <strong>Rollbar</strong>! Create an account and get special credits at <a href=""></a></p> <p>Guest co-host: <a href="">Miguel Grinberg</a></p> <p><strong>Miguel #1:</strong> <a href=""><strong>lolviz</strong></a></p> <ul> <li>Generates graphical representations of Python data structures using graphviz.</li> <li>Great as a teaching tool!</li> <li>Currently supports dicts, lists, lists of lists, linked lists and binary trees.</li> <li>Jupyter knows how to render these graphics. In regular Python it can also be used, but it is a bit cumbersome.</li> <li>I hope the project grows to support more complex data structures in the future!</li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>Odo for data transforms</strong></a></p> <ul> <li>Odo migrates between many formats.</li> </ul> <pre><code>odo(df, list) # create new list from Pandas DataFrame odo(df, []) # append onto existing list odo(df, 'myfile.json') # Dump dataframe to line-delimited JSON odo('myfiles.*.csv', Iterator) # Stream through many CSV files odo(df, 'postgresql://hostname::tablename') # Migrate dataframe to Postgres odo('myfile.*.csv', 'postgresql://hostname::tablename') # Load CSVs to Postgres odo('postgresql://hostname::tablename', 'myfile.json') # Dump Postgres to JSON odo('mongodb://hostname/db::collection', pd.DataFrame) # Dump Mongo to DataFrame </code></pre> <p><strong>Miguel #3:</strong> <a href=""><strong>Python Concurrency From the Ground Up</strong></a></p> <ul> <li>This is probably my favorite tech talk of all times.</li> <li>There are no slides, the entire talk is a live coding session.</li> <li>David Beazley covers concurrency with threads and processes, and then goes on to build an asynchronous framework along the lines of asyncio just using generators, all in front of your eyes.</li> <li>If you spend 45 minutes watching this talk you’ll end up with a much better understanding of Python concurrency.</li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>FAT Python : the next chapter in Python optimization</strong></a></p> <ul> <li>via Anthony Shaw</li> <li>The FAT Python project was started by Victor Stinner in October 2015 to try to solve issues of previous attempts of “static optimizers” for Python.</li> <li>The PEPs <ul> <li>PEP 511 is a proposal to add a process to optimize an AST instance. The AST instance is a object-oriented representation of your code. </li> <li>A bespoke optimizer could look at a set of domain specific changes, e.g. NumPy or Pandas “anti-patterns” and optimize them in the syntax tree. In replacement of a static linter that simply recommends changes, the optimizer could make those changes for you.</li> <li>PEP 509: Python is hard to optimize because almost everything is mutable: builtin functions, function code, global variables, local variables, … can be modified at runtime.</li> <li>The speedup of optimizations depends on the speed of guard checks. PEP 509 proposes to add a private version to dictionaries to implement fast guards on namespaces.</li> <li>PEP 510 proposes to add a public API to the Python C API to add specialized codes with guards to a function. When the function is called, a specialized code is used if nothing changed, otherwise use the original bytecode.</li> </ul></li> <li>Can download and compile this variation of CPython</li> <li>Basic function with a return is 24% improvement over 3.6 (and 46% faster than 2.7)</li> <li>Combining these 3 PEPs, we could see both implementation of guards as well as well as a range of optimizers out on PyPi.</li> </ul> <p><strong>Miguel #5:</strong> <a href=""><strong>sshuttle</strong></a></p> <ul> <li>You probably know that there are security risks when going online at public wi-fi hotspots at coffee shops, hotels or airports.</li> <li>Most people don’t realize this, but even if you access sites over https://, DNS queries made to connect to those sites are not encrypted, so they give away which sites you visit.</li> <li>sshuttle is fantastic tool (written in Python, BTW) that creates a secure tunnel between your machine and another machine (which can be in a secure location such as your home or office) and forwards all network traffic through that other system with strong encryption. A poor man’s VPN!</li> <li>All you need to use sshuttle is SSH access to the secure system. No need to install anything on the remote system besides SSH!</li> <li>Simply run <strong>sshuttle --dns --r username@your-server</strong> and from then on all traffic will be tunneled to your secure server with strong encryption, including DNS queries!</li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>Node.js forks again – this time it's a war of words over codes of conducts</strong></a></p> <ul> <li>After years of battling a string of systematic failures of governance and leadership, the Node.js community reached a breaking point.</li> <li>Monday saw a stream of resignations, one after the other throughout the day from Node.js' technical steering committee (TSC), a group that manages the day-to-day governance for the Node.js project.</li> <li>A third of the committee had quit their positions by the end of the day, including its first woman member.</li> <li>One person has left the project entirely.</li> <li>The resignations followed a single event -- a vote that failed to remove a former director, a longstanding member of the community, from the leadership group. Many of the complaints, since removed from the committee's pages, document a litany of violations of the community's code of conduct.</li> <li>The failure to have him removed from the position was seen as the embodiment of years of efforts to reform a pattern of harmful behaviors that was tearing the community apart.</li> <li>The inability for members of the TSC to "look at the entire picture" of a person's behavior rather than each broken rule is where trust in the system broke down, Kapke said.</li> <li>Moments after the failed leadership vote, Kat Marchán pushed the button that created Ayo.js, a new open-source project forked from Node.js.</li> </ul> <p><strong>Our news</strong></p> <p>Miguel: </p> <ul> <li><a href="">Blog</a> </li> <li><a href="">The New and Improved Flask Mega-Tutorial</a></li> </ul> <p>Michael: </p> <ul> <li><a href="">RESTful and HTTP APIs in Pyramid</a></li> </ul>
Aug 31, 2017
#40 Packet Manipulation with Scapy
<p>We have guest hosts filling in for Michael while he is on vacation. This week we have <a href="">Eric Chou</a>, author of the book “Mastering Python Networking” and a self-proclaimed Network Automation Nerd. </p> <p><strong>Eric #1:</strong> <a href=""><strong>DevOps Automation Tool: Ansible</strong></a></p> <ul> <li>DevOps Automation framework written in Python, code hosted on <a href="">GitHub</a>.</li> <li><a href="">Top 10 OpenSource projects in 2014 by</a>, along with Docker, Kubernetes, Apache Hadoop, OpenStack, and OpenDaylight, etc. </li> <li>Excellent <a href="">documentation</a> for all modules. </li> <li>Agentless, ‘networking vendor’ friendly, execute code locally that interacts with the device via SSH and API. </li> <li><a href="">Lots of Network modules</a>, including Cisco, Juniper, Arista, etc. In fact, you can find Cisco and Juniper testimonial on the Ansible site. </li> <li>Easy to learn and extend if you already know a little bit about Python, YAML, and Jinja2. </li> </ul> <p><strong>Brian #2:</strong> <a href=""><strong>Python Practices for Efficient Code: Performance, Memory, and Usability</strong></a></p> <p>(I’m too opinionated to leave out my thoughts when covering this article, even though it’s very well written and I mean no disrespect to Satwik Kansal)</p> <ol> <li>Try not to blow off memory <ul> <li>use generators to calculate large sets of results</li> <li>for big number crunching, use <a href="">numpy</a></li> <li>Use format instead of + for large strings. (or f-strings - Brian)</li> <li>Use slots for classes (psshh, use attrs - Brian)</li> </ul></li> <li>Python 2 or 3 <ul> <li>Write code compatible with both. (disagree, use 3 unless you can’t for a very good reason, then write code that’s easy to convert to 3 later. - Brian)</li> </ul></li> <li>Write Beautiful code because “The first impression is the last impression." <ul> <li>follow style guides</li> <li>use static analysis tools. Recommended using something called coala that’s installed as “coala-bears.</li> <li>(Brian: Maintenance cost is a real thing. Make your code look good because it’s cheaper in the long run. Use pycodestyle, pydocstyle, flake8, and if using sublime, use <a href="">Flake8Lint</a>) </li> </ul></li> <li>Speed up your performance <ul> <li>Multiprocess, not Multi-thread</li> </ul></li> <li>Analyzing your code <ul> <li>Use cProfile, memory_profiler, objgraph, resource</li> </ul></li> <li>Testing and CI <ul> <li>nose or pytest or doctest</li> <li>(Brian: BTW, I really appreciate the links to for tutorials on these.)</li> <li>(Brian: No. Use pytest)</li> <li>measure coverage and and try for 100%</li> <li>(Brian: No. use coverage to be alerted of sudden changes, and of code that possibly needs more testing and/or deleted)</li> </ul></li> </ol> <p><strong>Eric #3:</strong> <a href=""><strong>Packet Manipulation Program: Scapy</strong></a></p> <ul> <li>Free Python-based interactive packet manipulation program and library, <a href="">GitHub</a>. </li> <li>Craft the packet from the ground up, you can use it to decode packets or craft packets. </li> <li>You are in control instead of limited to what the creator of the tool can imagine, i.e. hping3, curl. </li> <li><a href="">Can be used together with the Python interpreter</a>. </li> <li>Particularly useful for network security <ul> <li>Crafting common attacks: malformed packets (such as IP version 3), Ping of Death (large paylaod), Land Attack (redirect the client response back to the client itself) for denial-of-service. </li> <li>Penetration Testing (TCP port scan) and Fuzzing by providing invalid, unexpected, or random data. </li> </ul></li> </ul> <p><strong>Brian #4:</strong> <a href=""><strong>Using Headless Chrome with Selenium</strong></a></p> <ul> <li>Miguel Grinberg quick demo of using headless chrome with selenium and unittest.</li> <li>(Brian: Eventually I’ll get Miguel to use pytest more.)</li> <li>Replace the normal Firefox with Chrome in the webdriver of Selenium, and passing a ‘headless’ argument to make it so the window doesn’t keep popping up and down when testing.</li> </ul> <p><strong>Eric</strong> <strong>#5:</strong> <a href=""><strong>Graph Visualization with Graphviz</strong></a></p> <ul> <li>Open Source graph visualization software. </li> <li>Perfect for graphing the large datacenter topology automatically or any other network diagrams. </li> <li>Extensive <a href="">documentation</a> and <a href="">gallery of examples</a>. </li> <li>Did I mention this is ‘automatible’? Thus avoid drifts between reality and actual network. </li> <li>Python package <a href="">graphviz</a> (lower case g) for Graphviz integration. </li> </ul> <p><strong>Brian</strong> <strong>#6:</strong> <a href=""><strong>PyCascades CFP still open until the 28th</strong></a></p> <ul> <li>Python conference in Vancouver, BC.</li> <li>Talks Jan 22, 23, Sprints Jan 24th</li> <li>Speakers get free admission. Talks are all 25 min slots. No Q&amp;A after talks in front of full audience, but speakers will hang out up front for a few minutes for individual questions</li> <li>I’m going to submit at least one proposal. But I’m kinda swamped this week, so the proposal will unfortunately be rushed.</li> </ul> <p><strong>Extra Eric:</strong></p> <ul> <li><a href=";fpl=fresh&amp;pd_rd_i=1784397008&amp;pd_rd_r=BQZKFJ3QVF5A5T1ANVZ0&amp;pd_rd_w=B4iFp&amp;pd_rd_wg=7ypBL&amp;pf_rd_m=ATVPDKIKX0DER&amp;pf_rd_s=desktop-1&amp;pf_rd_r=TBZN1MW3TEJYFE86QJ3N&amp;pf_rd_r=TBZN1MW3TEJYFE86QJ3N&amp;pf_rd_t=36701&amp;pf_rd_p=781f4767-b4d4-466b-8c26-2639359664eb&amp;pf_rd_p=781f4767-b4d4-466b-8c26-2639359664eb&amp;pf_rd_i=desktop&amp;linkCode=li3&amp;tag=pythfornetwen-20&amp;linkId=352f479d902e9d968e5d1832619de63e">Mastering Python Networking</a> book </li> <li>Network Labs: <a href="">Cisco Virtual Internet Routing Lab (VIRL)</a>, <a href="">Cisco DevNet</a>, <a href="">GNS3 (Graphic Network Simulator)</a>. </li> </ul> <p><strong>Extra Brian:</strong></p> <ul> <li>Copy editing and final testing with most recent Python and pytest done for <a href="">Python Testing with pytest</a></li> </ul>
Aug 24, 2017
#39 The new PyPI
<p><strong>Mahmoud #1:</strong> <a href=""><strong>The New PyPI</strong></a></p> <ul> <li>Donald Stufft and his PyPA team have been hard at work replacing the old</li> <li>The new site is now handling almost all the old functionality (excepting deprecated features, of course): <a href=""></a></li> <li>The new site has handled downloads (presently exceeding 1PB monthly bandwidth) for a while now, and uploads as of recently.</li> <li>A nice full-fledged, open-source Python application, eagerly awaiting your review and contribution: <a href=""></a></li> <li>More updates at: <a href=""></a></li> </ul> <p><strong>Brian #2:</strong> <a href=""><strong>CircuitPython Snakes its Way onto Adafruit Hardware</strong></a></p> <ul> <li><a href="">Adafruit announced CircuitPython in January</a> <ul> <li>“CircuitPython is based on the <a href="">open-source</a> <a href="">MicroPython</a> which brings the popular Python language to microcontrollers. The goal of CircuitPython is to make hardware as simple and easy as possible.”</li> <li>Already runs on <a href="">Metro M0 Express</a>, <a href="">Feather M0 Express</a>, and they are working on support for <a href="">Circuit Playground Express</a>, and now Gemma M0</li> </ul></li> <li>New product is <a href="">Gemma M0</a>: <ul> <li><a href="">Announced</a> at the end of July.</li> <li>It’s about the size of a quarter and is considered a wearable computer.</li> <li>“When you plug it in, it will show up as a very small disk drive with <strong></strong> on it. Edit <strong></strong> with your favorite text editor to build your project using Python, the most popular programming language. No installs, IDE or compiler needed, so you can use it on any computer, even ChromeBooks or computers you can’t install software on. When you’re done, unplug the Gemma M0 and your code will go with you."</li> <li>They’re under $10. I gotta get one of these and play with it. (Anyone from Adafruit listening, want to send me one?)</li> <li>Here's the intro video for it: <a href=";">;</a></li> </ul></li> <li><a href="">Creating and sharing a CircuitPython Library</a> is a good introduction to the Python open source community, including: <ul> <li>Creating a library (package or module)</li> <li>Sharing on GitHub</li> <li>Sharing docs on ReadTheDocs</li> <li>Testing with Travis CI</li> <li>Releasing on GitHub</li> </ul></li> </ul> <p><strong>Mahmoud #3:</strong> <strong>Dataclasses</strong></p> <ul> <li>Python has had classes for a long time, but maybe it’s time for some updated syntax and semantics, something higher level perhaps?</li> <li>dataclasses is an interesting case of Python’s core dev doing their own take on community innovation (Hynek’s attrs:</li> <li>Code, issues, and draft PEP at</li> </ul> <p><strong>Brian #4:</strong> <a href=""><strong>Pandas in a Nutshell</strong></a></p> <ul> <li>Jupyter Notebook style post. Tutorial by example with just a bit of extra text for explanation.</li> <li>Data structures: <ul> <li>Series – it’s a one dimensional array with indexes, it stores a single column or row of data in a Dataframe</li> <li>Dataframe – it’s a tabular spreadsheet like structure representing rows each of which contains one or multiple columns</li> </ul></li> <li>Series: Custom indices, adding two series, naming series, …</li> <li>Dataframes: using .head() and .tail(), info(), adding columns, adding a column as a calculation of another column, deleting a column, creating a dataframe from a dictionary, reindexing, summing columns and rows, .describe() for simple statistics, corr() for correlations, dealing with missing values, dropping rows, selecting, sorting, multi-indexing, grouping, </li> </ul> <p><strong>Mahmoud</strong> <strong>#5:</strong> <strong>Static Typing</strong></p> <ul> <li>PyBay 2017, which ended a day before recording, featured a neat panel on static typing in Python.</li> <li>One member each from Google, Quora, PyCharm, Facebook, and University of California</li> <li>Three different static analysis tools (four, if you count PyLint)</li> <li>They’re all collaborating already, and open to much more, as we can see on this collection of the stdlib’s type defs: <a href=""></a></li> <li>A fair degree of consensus around static types being most useful for testable documentation, like doctests, but with more systemic implications</li> <li>Not intended to be an algebraic type system (like Haskell, etc.)</li> </ul> <p><strong>Brian</strong> <strong>#6:</strong> <a href=""><strong>Full Stack Python Explains ORMs</strong></a></p> <ul> <li>What are Object Relational Mappers? <ul> <li>“An object-relational mapper (ORM) is a code library that automates the transfer of data stored in relational databases tables into objects that are more commonly used in application code.”</li> </ul></li> <li>Why are they useful? <ul> <li>“ORMs provide a high-level abstraction upon a relational database that allows a developer to write Python code instead of SQL to create, read, update and delete data and schemas in their database.”</li> </ul></li> <li>Do you need to use them?</li> <li>Downsides to ORMs: <ul> <li>Impedance mismatch : “the way a developer uses objects is different from how data is stored and joined in relational tables”</li> <li>Potential for reduced performance: code in the middle isn’t free</li> <li>Shifting complexity from the database into the application code : people usually don’t use database stored procedures when working with ORMs.</li> </ul></li> <li>A handful of popular ones including Django ORM, SQLAlchemy, Peewee, Pony, and SQLObject. Mostly listed as pointing out that they are active projects, brief description, and links for more info.</li> <li>Matt also has a <a href="">SQLAlchemy page</a> and a <a href="">peewee page</a> for more info on them.</li> </ul> <p><strong>Extra Mahmoud:</strong></p> <ul> <li><a href="">hyperlink</a></li> <li><a href=""></a> + <a href=""></a><a href="">(server code in Python)</a></li> </ul> <p><strong>Extra Brian:</strong></p> <ul> <li><a href="">Python Testing with pytest</a> has a <a href="">Discussion Forum</a>. It’s something that I think all Pragmatic books have. Just this morning I answered a question about the difference between monkeypatch and mock and when you would use one over the other.</li> </ul>
Aug 17, 2017
#38 Hacking Classic Nintendo Games with Python
<p><strong>Matt #1: <a href="">Hacking Classic Nintendo Games with Python</a> <a href=""></a></strong></p> <ul> <li>Gist: used the FCEUX (<a href=""></a>) Nintendo emulator’s debugger to hex edit memory and change what’s happening during play</li> <li>Hex changing is how the old school Game Genie worked</li> <li>Given by my Twilio colleague Sam Agnew at PyCon 2017, and all the talks are up on YouTube</li> <li>Sam was inspired by Guto Maia’s PyNES: <a href=""></a></li> <li>Sam uses the Lua programming language to automate changing the Mario and Zelda’s hex values. </li> <li>He then creates a Flask app where PyCon attendees could send a text message containing a hex address and 2 digit hex value to a phone number. the input would then be read into the game as he was playing</li> <li>What I particularly enjoyed about this talk is that it takes a bunch of topics that sound really complicated, like hex editing memory values, and makes it more accessible to less experienced developers because they can see the results</li> <li>Follow along with this blog post: <a href=""></a></li> </ul> <p><strong>Brian</strong> <strong>#2</strong>: <a href=""><strong>The Pac-Man Rule at Conferences</strong></a></p> <ul> <li>by Eric Holscher</li> <li>“When standing as a group of people, always leave room for 1 person to join your group.”</li> <li>“Leaving room for new people when standing in a group is a physical way to show an inclusive and welcoming environment. “</li> </ul> <p><strong>Matt #3:</strong> <strong>Bokeh</strong></p> <ul> <li>Python data visualization library where the visualization output is designed for presentation in web browsers</li> <li>Just released v0.12.6 in June, which has a slew of improvements. awesome development team and constantly improving</li> <li>v0.12.6 is last planned release before 1.0</li> <li>Wide range of visualizations you can create with Bokeh, including classic ones just bar charts box plots, and also interactive visuals</li> <li>Basically if you thought d3.js visualizations were awesome but didnt want to spend that much time hand crafting some complicated JavaScript, Bokeh will be your jam</li> <li>Flask-based tutorial: <a href=""></a></li> </ul> <p><strong>Brian #4:</strong> <a href=""><strong>Mosh (mobile shell)</strong></a></p> <ul> <li>Persuasive video: <a href=""></a> from 2012</li> <li>From the main page: <ul> <li>Remote terminal application that allows <strong>roaming</strong>, supports <strong>intermittent connectivity</strong>, and provides intelligent <strong>local echo</strong> and line editing of user keystrokes.</li> <li>Mosh is a replacement for SSH. It's more robust and responsive, especially over Wi-Fi, cellular, and long-distance links.</li> <li>Mosh is free software, available for GNU/Linux, BSD, macOS, Solaris, Android, Chrome, and iOS.</li> </ul></li> <li>This has been around since 2012. I just heard of it. Are people using it?</li> </ul> <p><strong>Matt</strong> <strong>#5:</strong> <a href=""><strong>Pelican static site generator</strong></a></p> <ul> <li>Static site generators take in a markup format such as reStructuredText or Markdown, along with a template engine such as Jinja and output HTML (or XML, JSON, etc) files that can be hosted anywhere</li> <li>It’s kind of a throw back to the early days of the web when everything was snappy</li> <li>Major new version 3.7.0 released at the end of 2016 with a minor v3.7.1 bump released early this year</li> <li>Lots of improvements to Python 3 compatibility. I use Pelican with Python 3.6.2. exclusively now.</li> <li>Significant customization by changing the configuration files.</li> <li>Lots of folks think static site generators are just for blogs, which is what most of the original static generators were built to create, but you really can create any type of site, including single page apps (when you combine a static site generator with a front end JavaScript framework).</li> <li>Just wrote a getting started tutorial: <a href="">How to Create Your First Static Site with Pelican and Jinja2</a></li> </ul> <p><strong>Brian</strong> <strong>#6:</strong> <a href=""><strong>pytest-watch</strong></a></p> <ul> <li><a href="">pytest 3.2.0</a> was released recently.</li> <li>Great for pytest users. Bummer for me that just recently tested all the code examples in the <a href="">Python Testing with pytest</a> book against pytest 3.1.3.</li> <li>So I wrote a bunch of tests to check every invocation of pytest in the book.</li> <li>I’m running it against both pytest 3.1.3 and pytest 3.2.0</li> <li>I’m automating this by running both versions every time I save a new test with pytest-watch</li> </ul> <pre><code> $ pip install pytest-watch $ cd &lt;test directory&gt; $ ptw . </code></pre> <ul> <li>Run <code>ptw .</code> in two windows, each with a virtualenv with different pytest versions, and I can test both constantly as I save tests.</li> <li>Will later convert this to tox, but for now, this is a huge timesaver.</li> </ul> <p><strong>(bonus) Matt #7: <a href="">Twilio Voices</a></strong></p> <ul> <li>New program where you get paid $500 for each published technical blog post you write for the Twilio blog. Every post has the code and walks the reader through how to recreate something you built.</li> <li>Examples: <a href="">Wedding at Scale</a>, <a href="">How I Hack My University Registration System</a></li> <li>Tell stories with code</li> <li>We put each post through a rigorous outline, voice and tech review process</li> <li>Doesn’t have to use Twilio, so you can write a post on pytest-watch, Mosh, Pelican, Bokeh, or any other library you’ve been meaning to work with and get paid when the post is published</li> <li>This is what I’ve been working on at Twilio for the past couple of months so we’ll work directly together on the posts</li> </ul>
Aug 09, 2017
#37 Rule over the shells with Sultan
<p><strong>Brian #1:</strong> <a href=""><strong>New URL for Python Developer’s Guide</strong></a></p> <ul> <li>How to contribute to CPython</li> </ul> <p>Some really useful links that I hadn’t noticed before. Also great ideas to include in a contributing guide for any large open source project:</p> <ul> <li>Core developers and contributors alike will find the following guides useful: <ul> <li><a href="">How to Contribute to Open Source</a> (from</li> <li><a href="">Building Welcoming Communities</a> (from</li> </ul></li> <li>Guide for contributing to Python: <ul> <li><a href="">Getting Started</a></li> <li><a href="">Where to Get Help</a></li> <li><a href="">Lifecycle of a Pull Request</a></li> <li><a href="">Running &amp; Writing Tests</a></li> <li>Beginner tasks to become familiar with the development process</li> <li><a href="">Helping with Documentation</a></li> <li><a href="">Increase Test Coverage</a></li> <li>Advanced tasks for once you are comfortable</li> <li><a href="">Silence Warnings From the Test Suite</a></li> <li>Fixing issues found by the <a href="">buildbots</a></li> <li><a href="">Fixing “easy” Issues (and Beyond)</a></li> <li><a href="">Using the Issue Tracker</a> and <a href="">Helping Triage Issues</a></li> <li><a href="">Triaging an Issue</a></li> <li><a href="">Experts Index</a></li> <li><a href="">Following Python’s Development</a></li> <li><a href="">How to Become a Core Developer</a></li> <li><a href="">Committing and Pushing Changes</a></li> <li><a href="">Development Cycle</a></li> <li><a href="">Continuous Integration</a></li> <li><a href="">Git Bootcamp and Cheat Sheet</a></li> </ul></li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>Sultan: Command and Rule Over Your Shell</strong></a></p> <ul> <li>Python package for interfacing with command-line utilities, like yum, apt-get, or ls, in a Pythonic manner</li> </ul> <p>Simple example</p> <pre><code>from sultan.api import Sultan s = Sultan() s.sudo("yum install -y tree").run() </code></pre> <p>Better in a context manager:</p> <pre><code>from sultan.api import Sultan with Sultan.load(sudo=True) as s: s.yum("install -y tree").run() </code></pre> <p>Even works remotely:</p> <pre><code>from sultan.api import Sultan with Sultan.load(sudo=True, hostname="") as sultan: sultan.yum("install -y tree").run() </code></pre> <p><strong>Brian #3:</strong> <a href=""><strong>Flake8Lint</strong></a></p> <ul> <li>Sublime Text plugin for lint Python files.</li> <li>Includes these linters and style checkers: <ul> <li><a href=""><strong>Flake8</strong></a> (used in "Python Flake8 Lint") is a wrapper around these tools:</li> <li><a href=""><strong>pep8</strong></a> is a tool to check your Python code against some of the style conventions in <a href="">PEP8</a>.</li> <li><a href=""><strong>PyFlakes</strong></a> checks only for logical errors in programs; it does not perform any check on style.</li> <li><a href=""><strong>mccabe</strong></a> is a code complexity checker. It is quite useful to detect over-complex code. According to McCabe, anything that goes beyond 10 is too complex. See <a href="">Cyclomatic_complexity</a>.</li> <li>There are additional tools used to lint Python files:</li> <li><a href=""><strong>pydocstyle</strong></a> is a static analysis tool for checking compliance with Python <a href="">PEP257</a>.</li> <li><a href=""><strong>pep8-naming</strong></a> is a naming convention checker for Python.</li> <li><a href=""><strong>flake8-debugger</strong></a> is a flake8 debug statement checker.</li> <li><a href=""><strong>flake8-import-order</strong></a> is a flake8 plugin that checks import order in the fashion of the Google Python Style Guide (turned off by default).</li> </ul></li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>Magic Wormhole</strong></a></p> <ul> <li>Get things from one computer to another, safely.</li> <li>A library and a command-line tool named <code>wormhole</code>, which makes it possible to get arbitrary-sized files and directories (or short pieces of text) from one computer to another.</li> <li>The two endpoints are identified by using identical "wormhole codes”</li> <li>Video from PyCon 2016: <a href=""></a></li> <li>The codes are short and human-pronounceable, using a phonetically-distinct wordlist.</li> <li>As a library too: The wormhole module makes it possible for other applications to use these code-protected channels. </li> </ul> <p><strong>Brian #5:</strong> <a href=""><strong>Python Virtual Environments Primer</strong></a></p> <ul> <li>why do we need virtual environments</li> <li>what are they</li> <li>how to use them / how do they work</li> <li>also <ul> <li>virtualenvwrapper</li> <li>using different versions of python</li> <li>pyvenv</li> </ul></li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>How Rust can replace C, with Python's help</strong></a></p> <ul> <li>Why Rust? Rust has <ul> <li>a type system feature that helps eliminate memory leaks,</li> <li>proper interfaces, called 'traits',</li> <li>better type inference,</li> <li>better support for concurrency,</li> <li>(almost) first-class functions that can be passed as arguments.</li> </ul></li> <li>It isn’t difficult to expose Rust code to Python. A Rust library can expose a C ABI (application binary interface) to Python without too much work. </li> <li>Some Rust crates (as Rust packages are called) already expose Python bindings to make them useful in Python.</li> <li>A new spate of projects are making it easier to develop Rust libraries with convenient bindings to Python – and to deploy Python packages that have Rust binaries</li> <li><a href=""><strong>Rust-CPython</strong></a><strong>:</strong> <ul> <li><strong>What it is:</strong> A set of bindings in Rust for the CPython runtime. This allows a Rust program to connect to CPython, use its ABI, run Python programs through it, and work with representations of Python objects in Rust itself.</li> <li><strong>Who it’s for:</strong> Rust programmers who want to hook into CPython and control it from the inside out.</li> </ul></li> <li><a href=""><strong>PyO3</strong></a> <ul> <li><strong>What it is:</strong> For Rust developers, the PyO3 project provides a basic way to write Rust software with bindings to Python in both directions. A Rust program can interface with Python objects and the Python interpreter, and can expose Rust methods to a Python program in the same way a C module does.</li> <li><strong>Who it’s for:</strong> Those writing modules that work closely with the Python runtime, and need to interact directly with it.</li> </ul></li> <li><a href=""><strong>Snaek</strong></a> <ul> <li><strong>What it is:</strong> Another project in the early stages, Snaek lets developers create Rust libraries that are loaded dynamically into Python as needed, but don’t rely on being linked statically against Python’s runtime.</li> <li>Doesn’t use CTypes but CFFI</li> <li><strong>Who it’s for:</strong> Those who want to expose methods written in Rust to a Python script, or for Rust developers who don’t want or need to become familiar with Python.</li> </ul></li> <li>And there is a cookiecutter project / template too <ul> <li><a href=""></a></li> <li>“A very important goal of the project,” writes its maintainers, “is that it be able to produce a binary distribution (Wheel) which will not require the end user to actually compile the Rust code themselves.”</li> </ul></li> </ul>
Aug 02, 2017
#36 Craft Your Python Like Poetry and Other Musings
<p>Brought to you by <strong>Rollbar</strong>! Create an account and get special credits at <a href=""></a></p> <p><strong>Brian #1:</strong> <a href=""><strong>Craft Your Python Like Poetry</strong></a></p> <ul> <li>Line length is important. Shorter is often more readable.</li> <li>line break placement makes a huge difference in readability and applies to <ul> <li>comprehensions</li> <li>function call parameters</li> <li>chained function calls. (Dot alignment is pleasing and nothing I have considered previously)</li> <li>dictionary literals</li> </ul></li> </ul> <p><a href=""></a> <strong>Michael #2:</strong> <a href=""><strong>The Fedora Python Classroom Lab</strong></a></p> <ul> <li>Makes it easy for teachers and instructors to use Fedora in their classrooms or workshops.</li> <li>Ready to use operating system with important stuff pre-installed</li> <li>With GNOME or as a headless environment for Docker or Vagrant</li> <li>Lots of prebuilt goodies, especially around data science: <ul> <li>IPython, Jupyter Notebook, multiple Pythons, virtualenvs, tox, git and more</li> </ul></li> </ul> <p><strong>Brian #3:</strong> <a href=""><strong>How a VC-funded company is undermining the open-source community</strong></a></p> <ul> <li>A San Francisco startup called Kite is being accused of underhanded tactics.</li> <li>An Atom plugin called Minimap, downloaded more than 3.5 M times, open source, and developed primarily by one person. @abe33</li> <li>abe33 hired by Kite, then adds a “Kite Promotion” “feature” to Minimap that examines user code and inserts links to related parts of Kite website. (Presumably in the minimap?)</li> <li>Users rightfully ticked.</li> <li>Next. autocomplete-Python, also an Atom addon, seems to be taken over by Kite engineers and changes the autocomplete from local Jedi engine to cloud Kite engine (also therefore sending users code to Kite). </li> <li>Seems like that ought to have been a separate plugin, not a take over of an existing one.</li> <li>Again, users not exactly supportive of the changes.</li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>Newspaper Python Package</strong></a></p> <ul> <li>News, full-text, and article metadata extraction in Python 3</li> <li>Behold the example code:</li> </ul> <pre><code> from newspaper import Article url = '' article = Article(url) article.parse() article.authors # ['Leigh Ann Caldwell', 'John Honway'] article.publish_date # datetime.datetime(2013, 12, 30, 0, 0) article.nlp() article.keywords # ['New Years', 'resolution', ...] article.summary # 'The study shows that 93% of people ...' </code></pre> <p><strong>Brian #5:</strong> <a href=""><strong>IEEE Spectrum:</strong></a> <a href=""><strong>The Top Programming Languages 2017</strong></a></p> <ul> <li>We’re #1. We’re #1.</li> <li>Python on top of the list</li> <li>IEEE very open about <a href="">their methodology.</a> <ul> <li>Combo of Google, Google Trends, GitHub, Twitter, Reddit, StackOverflow, HackerNews, CareerBuilder, Dice, and IEEE Xplore Digital Library</li> </ul></li> <li>Python #1 in lots of categories. Java still has more job openings, supposedly. Although I think it’s because Java people are quitting to go work on Python projects. </li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>SciPy 2017 videos are out</strong></a></p> <ul> <li>Bunch of tutorials</li> <li>Keynote - Coding for Science and Innovation, Gaël Varoquaux</li> <li>Dash - A New Framework for Building User Interfaces for Technical Computing, </li> <li>Dask - Advanced Techniques, Matthew Rocklin</li> <li>Scientific Analysis at Scale - a Comparison of Five Systems, Jake V.</li> <li>Keynote - Academic Open Source, Kathryn Huff</li> <li>Plus lots more</li> </ul>
Jul 28, 2017
#35 How developers change programming languages over time
<p><strong>Brian #1:</strong> <a href=""><strong>Python Quirks</strong></a> <a href=""><strong>: Comments</strong></a></p> <ul> <li>Python developers put comments in their code.</li> </ul> <pre><code> # Like this """ And like this """ "And like this." ["Not usually like this","but it's possible"] </code></pre> <ul> <li>Philip Trauner timed all of these.</li> <li>Actual # comments are obviously way faster.</li> <li>He also shows the AST difference.</li> <li>Don’t abuse the language. Unused unreferenced strings are not free.</li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>Python 3.6.2 is out!</strong></a></p> <ul> <li><strong>Security</strong> <ul> <li>bpo-30730: Prevent environment variables injection in subprocess on Windows. Prevent passing other environment variables and command arguments.</li> <li>bpo-30694: Upgrade expat copy from 2.2.0 to 2.2.1 to get fixes of multiple security vulnerabilities including: CVE-2017-9233 (External entity infinite loop DoS), CVE-2016-9063 (Integer overflow, re-fix), CVE-2016-0718 (Fix regression bugs from 2.2.0’s fix to CVE-2016-0718) and CVE-2012-0876 (Counter hash flooding with SipHash). Note: the CVE-2016-5300 (Use os-specific entropy sources like getrandom) doesn’t impact Python, since Python already gets entropy from the OS to set the expat secret using XML_SetHashSalt().</li> <li>bpo-30500: Fix urllib.parse.splithost() to correctly parse fragments. For example, splithost('//') now correctly returns the host, instead of treating as the host in an authentification (login@host).</li> </ul></li> <li><strong>Core and Builtins</strong> <ul> <li>bpo-29104: Fixed parsing backslashes in f-strings.</li> <li>bpo-27945: Fixed various segfaults with dict when input collections are mutated during searching, inserting or comparing. Based on patches by Duane Griffin and Tim Mitchell.</li> <li>bpo-30039: If a KeyboardInterrupt happens when the interpreter is in the middle of resuming a chain of nested ‘yield from’ or ‘await’ calls, it’s now correctly delivered to the innermost frame.</li> <li>Library</li> <li>bpo-30038: Fix race condition between signal delivery and wakeup file descriptor. Patch by Nathaniel Smith.</li> <li>bpo-23894: lib2to3 now recognizes rb'...' and f'...' strings.</li> <li>bpo-24484: Avoid race condition in multiprocessing cleanup (#2159)</li> </ul></li> <li><strong>Windows</strong> <ul> <li>bpo-30687: Locate msbuild.exe on Windows when building rather than vcvarsall.bat</li> <li>bpo-30450: The build process on Windows no longer depends on Subversion, instead pulling external code from GitHub via a Python script. If Python 3.6 is not found on the system (via py -3.6), NuGet is used to download a copy of 32-bit Python.</li> </ul></li> <li><strong>Plus about 40 more fixes / changes</strong></li> </ul> <p><strong>Brian #3:</strong> <a href=""><strong>Contributing to Open Source Projects: Imposter Syndrome Disclaimer</strong></a></p> <ul> <li>“How to contribute” often part of OSS projects.</li> <li>Adrienne Lowe of has an “Imposter Syndrome Disclaimer” to include in your contributing documentation that’s pretty great.</li> <li>She’s also <a href="">collecting examples</a> of people using it, or similar.</li> <li>From the disclaimer: </li> </ul> <blockquote> <p>“<em>Imposter syndrome disclaimer</em>: I want your help. No really, I do. There might be a little voice inside that tells you you're not ready; that you need to do one more tutorial, or learn another framework, or write a few more blog posts before you can help me with this project. I assure you, that's not the case. … And you don't just have to write code. You can help out by writing documentation, tests, or even by giving feedback about this work. (And yes, that includes giving feedback about the contribution guidelines.)“</p> </blockquote> <p><strong>Michael #4:</strong> <a href=""><strong>The Dark Secret at the Heart of AI</strong></a></p> <ul> <li>via MIT Technology Review</li> <li>There’s a big problem with AI: even its creators can’t explain how it works</li> <li>Last year, an experimental vehicle, developed by researchers at the chip maker Nvidia, didn’t look different from other autonomous cars, but it was unlike anything demonstrated by Google, Tesla, or General Motors, and it showed the rising power of artificial intelligence. The car didn’t follow a single instruction provided by an engineer or programmer. Instead, it relied entirely on an algorithm that had taught itself to drive by watching a human do it.</li> <li>The result seems to match the responses you’d expect from a human driver. But what if one day it did something unexpected—crashed into a tree, or sat at a green light? </li> <li>As things stand now, it might be difficult to find out why.</li> <li>And you can’t ask it: there is no obvious way to design such a system so that it could always explain why it did what it did.</li> <li>There’s already an argument that being able to interrogate an AI system about how it reached its conclusions is a fundamental legal right</li> <li>We’ve never before built machines that operate in ways their creators don’t understand. How well can we expect to communicate—and get along with—intelligent machines that could be unpredictable and inscrutable</li> </ul> <p><strong>Brian #5:</strong> <a href=""><strong>Arrange Act Assert pattern for Python developers</strong></a></p> <ul> <li>James Cooke</li> <li>Good introduction to test case structure.</li> <li>Split your tests into setup, action, assertions.</li> <li>Pattern also known by: <ul> <li>Given, When, Then</li> <li>Setup, Test, Teardown</li> <li>Setup, Exercise, Verify, Teardown</li> </ul></li> <li>Also covered in: <ul> <li><a href=""></a></li> <li><a href=""></a></li> </ul></li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>Analyzing GitHub, how developers change programming languages over time</strong></a></p> <ul> <li>From source{d}: Building the first AI that understands code</li> <li>Have you ever been struggling with an nth obscure project, thinking : “I could do the job with this language but why not switch to another one which would be more enjoyable to work with” ?</li> <li>Derived from <a href=""><strong>The eigenvector of “Why we moved from language X to language Y”</strong></a><strong>,</strong> <a href=""><strong>Erik Bernhardsson</strong></a> <em>*</em>*</li> <li>Dataset available <ul> <li>4.5 Million GitHub users</li> <li>393 different languages</li> <li>10 TB of source code in total</li> </ul></li> <li>I find it nice to visualize developer’s language usage history with a kind of <a href=""><strong>Gantt diagram</strong></a>.</li> <li>We did not include Javascript because</li> <li>Most popular languages on GitHub</li> <li>At last! Here is the reward: the stationary distribution of our Markov chain. This probability distribution is independent of the initial distribution. It gives information about the stability of the process of random switching between languages. </li> <li><table> <thead> <tr> <th>Rank</th> <th>Language</th> <th>Popularity, %</th> <th>Source code, %</th> </tr> </thead> <tbody> <tr> <td>1.</td> <td>Python</td> <td>16.1</td> <td>11.3</td> </tr> <tr> <td>2.</td> <td>Java</td> <td>15.3</td> <td>16.6</td> </tr> <tr> <td>3.</td> <td>C</td> <td>9.2</td> <td>17.2</td> </tr> <tr> <td>4.</td> <td>C++</td> <td>9.1</td> <td>12.6</td> </tr> <tr> <td>5.</td> <td>PHP</td> <td>8.5</td> <td>24.4</td> </tr> <tr> <td>6.</td> <td>Ruby</td> <td>8.3</td> <td>2.6</td> </tr> <tr> <td>7.</td> <td>C#</td> <td>6.1</td> <td>6.5</td> </tr> </tbody> </table></li> <li><p>Python (16.1 %) appears to be the most attractive language, followed closely by Java (15.3 %). It’s especially interesting since only 11.3 % of all source code on GitHub is written in Python.</p></li> <li>Although there are ten times more lines of code on GitHub in PHP than in Ruby, they have the same stationary distribution.</li> <li>What about sticking to a language ? <ul> <li>Developers coding in one of the 5 most popular languages (Java, C, C++, PHP, Ruby) are most likely to switch to Python with approx. 22% chance on average.</li> <li>Similarly, a Visual Basic developer has more chance (24%) to move to C# while Erik’s is almost sure in this transition with 92% chance.</li> <li>People using numerical and statistical environments such as Fortran (36 %), Matlab (33 %) or R (40 %) are most likely to switch to Python in contrast to Erik’s matrix which predicts C as their future language.</li> </ul></li> </ul>
Jul 19, 2017
#34 The Real Threat of Artificial Intelligence
<p>Sponsored by Rollbar! Get the bootstrap plan at <a href=""></a></p> <p><strong>Brian #1:</strong> <a href=""><strong>Easy Python logging with daiquiri</strong></a></p> <ul> <li>Standard library logging package is non-intuitive. </li> <li>Daiquiri is better.</li> <li>Logs to stderr by default.</li> <li>Use colors if logging to a terminal.</li> <li>Support file logging.</li> <li>Use program name as the name of the logging file so providing just a directory for logging will work.</li> <li>Support syslog.</li> <li>Support journald.</li> <li>JSON output support.</li> <li>Support of arbitrary key/value context information providing.</li> <li>Capture the warnings emitted by the warnings module.</li> <li>Native logging of any exception.</li> <li>This works:</li> </ul> <pre><code> import daiquiri daiquiri.setup() logger = daiquiri.getLogger() logger.error("something wrong happened") </code></pre> <ul> <li>Also check out <a href="">logzero</a></li> </ul> <p><code> from logzero import logger logger.debug("hello")"info") logger.warn("warn") logger.error("error") </code></p> <p><strong>Michael #2:</strong> <a href=""><strong>The Real Threat of Artificial Intelligence</strong></a></p> <ul> <li>What worries you about the coming world of artificial intelligence?</li> <li>Too often the answer to this question resembles the plot of a sci-fi thriller. People worry that developments in A.I. will bring about the “singularity”</li> <li>This doesn’t mean we have nothing to worry about. </li> <li>On the contrary, the A.I. products that now exist are improving faster than most people realize and promise to radically transform our world, not always for the better</li> <li>AI will reshape what work means and how wealth is created, leading to unprecedented economic inequalities and even altering the global balance of power</li> <li>This kind of A.I. is spreading to thousands of domains (not just loans), and as it does, it will eliminate many jobs. Bank tellers, customer service representatives, telemarketers, stock and bond traders, even paralegals and radiologists will gradually be replaced by such software.</li> <li>Part of the answer will involve educating or retraining people in tasks A.I. tools aren’t good at. Artificial intelligence is poorly suited for jobs involving creativity, planning and “cross-domain” thinking — for example, the work of a trial lawyer. </li> <li>The solution to the problem of mass unemployment, I suspect, will involve “service jobs of love.” These are jobs that A.I. cannot do, that society needs and that give people a sense of purpose. Examples include accompanying an older person to visit a doctor, mentoring at an orphanage</li> <li>This leads to the final and perhaps most consequential challenge of A.I. The Keynesian approach I have sketched out may be feasible in the United States and China, which will have enough successful A.I. businesses to fund welfare initiatives via taxes. But what about other countries?</li> </ul> <p><strong>Brian #3:</strong> <a href=""><strong>The three laws of config dynamics</strong></a></p> <ul> <li>The birth of configuration files</li> <li><strong>Law 1</strong> Config values can be transformed from one form to another, but can be neither created nor destroyed.</li> <li><strong>Law 2</strong> The total length of a config file can only increase over time.</li> <li><strong>Law 3</strong> The length of a perfect config file in a development environment is exactly equal to zero.</li> <li>Docker can help</li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>Five Tips To Get You Started With Jupyter Notebook</strong></a></p> <ul> <li>Don’t Put Your Entire Code in a Single Cell</li> <li>There are different types of cells</li> <li>Executing Cells (shift + enter)</li> <li>Explore Interactive Mapping Options (via ArcGIS)</li> <li>To explore new modules, use questions and TAB auto-complete (Object?)</li> </ul> <p><strong>Brian #5:</strong> <a href=""><strong>Cost of Coupling Versus Cost of De-coupling</strong></a></p> <ul> <li>Two elements are coupled wrt a given change iff changing one element implies changing the other.</li> <li>Decoupled code, or loosely coupled, follows DRY principles, uses smaller components, is more modular, etc. But also has more files, more classes, handles more cases, and takes longer to write.</li> <li>There is a place for both. </li> <li>Kent describes two phases, Explore and Extract.</li> <li>Explore <ul> <li>more learning</li> <li>tracer bullets, spike projects, first drafts, happy path implementation</li> <li>coupled code, copy/paste coding, etc work fine and are faster because the design and architecture aren’t the goal, learning is the goal</li> <li>answer questions quickly</li> <li>ask better questions based on learnings</li> </ul></li> <li>Extract <ul> <li>Candidate Release, final draft, architected</li> <li>Economies of scale take over</li> <li>Return on investment</li> <li>Minimize cost of changes as code base grows.</li> </ul></li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>100 Days of Code at PyBites</strong></a></p> <ul> <li>The Challenge: <a href="">Join the #100DaysOfCode</a></li> <li>Stats: <a href="">We wrote roughly 5K lines of code</a>, divided into 100 scripts, one each day</li> <li>We <a href="">auto-tweeted</a> our progress each day which was tracked in our <a href="">log file</a>.</li> <li>Module Index: We ended up using exactly 100 modules as well (weird coincidence LOL)</li> <li>Showcase of 10 Utilities</li> <li>The rumors are true: our next 100 days project will be around learning Django</li> </ul> <p>Extra:</p> <ul> <li>First book review of up, <a href=""></a></li> <li>Python for Entrepreneurs has officially launch! Over 19 hours of content. Get it at <a href=""></a></li> </ul>
Jul 13, 2017
#33 You should build an Alexa skill
<p>Sponsored by Rollbar! <a href=""></a> </p> <p><strong>Brian #1:</strong> <a href=""><strong>Linting as Lightweight Defect Detection for Python</strong></a></p> <ul> <li>flake8, </li> <li>pycodestyle, formerly pep8 tool <a href=""></a></li> <li>pep257 can be checked with flake8-docstrings</li> <li>pydocstyle, <a href=""></a></li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>You should build an Alexa skill</strong></a></p> <ul> <li>Jacqueline Wilson wrote <em>Amazon Alexa Skill Recipe with Python 3.6</em></li> <li>Ingredients: <ul> <li>A developer account on <a href=""></a> (“Amazon Developer Console”)</li> <li>An AWS account on <a href=""></a> (“AWS Console”)</li> <li>Beginner knowledge of Python 3.x syntax</li> </ul></li> <li>Create a “What’s for dinner” bot</li> <li>Amazon calls these utterances: <ul> <li>“What should I have for dinner?”</li> <li>“Do you have a dinner idea?”</li> <li>“What’s for dinner?”</li> </ul></li> <li>Tie the commands to an AWS Lambda function (returns a JSON response)</li> <li>Test via <a href="">Alexa Skill Testing Tool</a> </li> </ul> <p><strong>Brian #3:</strong> <a href=""><strong>RISE</strong></a></p> <ul> <li>Reveal IPython Slide Extension</li> <li>Making slides with Jupyter notebooks</li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>Closer</strong></a></p> <ul> <li>Run, monitor and close remote SSH processes automatically</li> <li>Closer was born because I had trouble with killing up processes I set up remotely via SSH. That is, you want to run some SSH process in the background, and then you want to kill it, just like you would a local subprocess.</li> <li>Main features: <ul> <li>kill the remote process (either by choice, or automatically at the end of the calling process)</li> <li>capture the remote process’s output</li> <li>live monitoring of remote process output</li> <li>get a callback upon remote process’ death</li> </ul></li> </ul> <p><strong>Brian #5:</strong> <a href=""><strong>Checklist for</strong></a> <a href=""><strong><em>*</a><a href=""></strong>Python libraries APIs</em>*</a></p> <p><strong>Michael #6:</strong> <a href=""><strong>Fades</strong></a></p> <ul> <li>Fades is a system that automatically handles the virtualenvs in the cases normally found when writing scripts and simple programs, and even helps to administer big projects.</li> <li>fades will automagically create a new virtualenv (or reuse a previous created one), installing the necessary dependencies, and execute your script inside that virtualenv, with the only requirement of executing the script with fades and also marking the required dependencies.</li> <li>At the moment you execute the script, fades will search a virtualenv with the marked dependencies, if it doesn’t exists fades will create it, and execute the script in that environment.</li> <li>Indicating dependencies (in code or via CLI)</li> </ul> <pre><code> import somemodule # fades == 3 import somemodule # fades &gt;= 2.1 import somemodule # fades &gt;=2.1,&lt;2.8,!=2.6.5 </code></pre> <ul> <li>Can control the Python version the env is based upon</li> <li>Can ask for a “refresh” on the virtual env</li> <li>You can also configure fades using .ini config files.</li> <li>How to clean up old virtualenvs?</li> </ul> <p><strong>Listener comment,</strong> <a href=""><strong>RE: Episode 32</strong></a><strong>:</strong></p> <p>Jan Oglop: </p> <p>Hello Michael and Brian, I wanted to thank you for amazing work you do. And let you know that you have helped me to find the working place from my dreams! My colleagues has similar hobbies and loves python as much as I do!</p> <p>Thank you again!</p>
Jul 06, 2017
#32 8 ways to contribute to open source when you have no time
<p><strong>Brian #1:</strong> <a href=""><strong>Introducing Dash</strong></a></p> <ul> <li>UI library for analytical web applications</li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>Keeping Python competitive</strong></a></p> <ul> <li>Article on LWN, interview with Victor Stinner</li> <li>He sees a need to improve Python performance in order to keep it competitive with other languages.</li> <li>Not as easy to optimize as other languages. For one thing, the C API blocks progress in this area</li> <li>Python 3.7 is as fast as Python 2.7 on most benchmarks, but 2.7 was released in 2010. Users are now comparing Python performance to that of Rust or Go, which had only been recently announced in 2010. </li> <li>In his opinion, the Python core developers need to find a way to speed Python up by a factor of two in order for it to continue to be successful.</li> <li>JITs may be part of the answer, notably Pyjion by Dino Viehland and Brett Cannon</li> <li>An attendee suggested Cython, which does AoT compilation, but its types are not Pythonic. He suggested that it might be possible to use the new type hints and Cython to create something more Pythonic.</li> </ul> <p><strong>Brian #3:</strong> <a href=""><strong>PyPI Quick and Dirty</strong></a></p> <ul> <li>A completely incomplete guide to packaging a Python module and sharing it with the world on PyPI. - Hynek Schlawack</li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>Minimal examples of data structures and algorithms in Python</strong></a></p> <ul> <li>Simple algorithmic examples in Python, including <ul> <li>linked lists</li> <li>reversing linked lists</li> <li>GCD</li> <li>Queues</li> <li>Binary search</li> <li>depth first search</li> <li>many, many more</li> </ul></li> </ul> <p><strong>Brian #5:</strong> <a href=""><strong>8 ways to contribute to open source when you have no time</strong></a></p> <p><strong>Michael #6:</strong> <a href=""><strong>NumPy receives first ever funding, thanks to Moore Foundation</strong></a></p> <ul> <li>For the first time ever, NumPy—a core project for the Python scientific computing stack—has received grant funding.</li> <li>The proposal, “<a href="">Improving NumPy for Better Data Science</a>” will receive $645,020 from the Moore Foundation over 2 years, with the funding going to UC Berkeley Institute for Data Science. </li> <li>The principal investigator is <a href="">Dr. Nathaniel Smith</a>.</li> <li>The NumPy project was started in 2006 by <a href="">Travis Oliphant</a>.</li> </ul>
Jul 01, 2017
#31 You should have a change log
<p><strong>Brian #1:</strong> <a href=""><strong>TinyMongo</strong></a></p> <ul> <li>Like MongoDB, but built on top of TinyDB.</li> <li>Even runs on a Raspberry Pi, according to Stephen</li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>A dead simple Python data validation library</strong></a></p> <ul> <li><code>validus.isemail('')</code> <ul> <li>Validation functions include:</li> </ul></li> <li>isrgbcolor()</li> <li>isphone()</li> <li>isisbn()</li> <li>isipv4()</li> <li>isint()</li> <li>isfloat()</li> <li>isslug()</li> <li>isuuid() <ul> <li>Requires Python 3.3+</li> </ul></li> </ul> <p><strong>Brian #3:</strong> <a href=""><strong>PuDB</strong></a></p> <ul> <li>In episode 29, <a href=""></a>, I talked about launching pdb from pytest failures.</li> <li><a href="">@kidpixo</a> pointed out that PuDB was a better debugger and can also be launched from pytest failures.</li> <li>Starting pudb from pytest failed tests (from <a href="">docs</a>): <code>pytest --pdbcls pudb.debugger:Debugger --pdb --capture=no</code></li> <li>Using <a href="">pytest-pudb</a> plugin to do the same: <code>pytest --pudb</code></li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>Analyzing Django requirement files on GitHub</strong></a></p> <ul> <li>From the guys</li> <li>Django is the most popular Python web framework. </li> <li>It is now almost 12 years old and is used on all kinds of different projects.</li> <li>Django developers pin their requirements (64%): Pinned or freezed requirements (Django==1.8.12) make builds predictable and deterministic.</li> <li>Django 1.8 is the most popular major release (24%) <ul> <li>A bit worrisome are the 1.9 (14%), 1.7 (13%) and 1.6 (13%) releases on the second, third and fourth place. All of them are no longer receiving security updates, 1.7 and 1.6 went EOL over 2 years ago.</li> </ul></li> <li>Yikes: Only 2% of all Django projects are on a secure release <ul> <li>Among all projects, more than 60% use a Django release with one or more known security vulnerabilities. Only 2% are using a secure Django release.</li> <li>On the remaining part of more than 30% it's unclear what exactly is going to be installed. That's because the Django release is either unpinned or has a range.</li> </ul></li> </ul> <p><strong>Brian #5:</strong> <strong>Changelogs</strong></p> <ul> <li><a href=""></a></li> <li><a href=""></a></li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>Understanding Asynchronous Programming in Python</strong></a></p> <ul> <li>by Doug Farrell via Dan Bader’s site</li> <li>A synchronous program is what most of us started out writing, and can be thought of as performing one execution step at a time, one after another.</li> <li>Example: A web server <ul> <li>Could be synchronous</li> <li>Could be fully optimized but</li> <li>You’re at best still waiting on network IO back to all the web clients</li> </ul></li> <li>The Real World is Asynchronous: <em>Kids are a long running task with high priority, superseding any other task we might be doing, like the checkbook or laundry</em>.</li> <li>Example 1: Synchronous Programming (using queuing)</li> <li>Example 2: Simple Cooperative Concurrency (using generators)</li> <li>Example 3: Cooperative Concurrency With Blocking Calls (same, but with slow operations)</li> <li>Example 4: Cooperative Concurrency With Non-Blocking Calls (gevent)</li> <li>Example 5: Synchronous (Blocking) HTTP Downloads</li> <li>Example 6: Asynchronous (Non-Blocking) HTTP Downloads With gevent</li> <li>Example 7: Asynchronous (Non-Blocking) HTTP Downloads With Twisted</li> <li>Example 8: Asynchronous (Non-Blocking) HTTP Downloads With Twisted Callbacks</li> </ul> <p>Errata/Giving Credit:</p> <ul> <li>Also in episode 29, <a href=""></a>, I talked about pipcache as an alias for pip download. I think I said the author of a blog post contacted me. It wasn’t him. It was <a href="">@kidpixo</a>. Sorry kidpixo, keep the ideas coming.</li> </ul> <p>For fun: Python Private Methods</p> <ul> <li><a href=""></a></li> </ul> <p>Our news</p> <ul> <li>Beta 3 of <a href="">Python Testing with pytest</a> should come out this week with Chapter 7: Using pytest with other tools, which includes using it with pdb,, mock, tox, and Jenkins. <ul> <li>Next beta will be the appendices, including a clean up and rewrite of pip and venv appendices, plus a plugin sampler pack, and a tutorial on packaging.</li> <li>Thanks to everyone who has submitted Errata. </li> </ul></li> <li>Finished recording RESTful and HTTP Services in Pyramid AND MongoDB for Python Developers. Add your email address at <a href=""></a> to get notified upon release of each.</li> </ul>
Jun 21, 2017
#30 You are not Google and other ruminations
<h1>Python Bytes 30</h1> <p>Sponsored by Datadog: Try Datadog and get a free shirt at <a href=""></a>.</p> <p><strong>Brian #1:</strong> <strong>Problems and Solutions are different at different scales</strong></p> <ul> <li><a href="">You are not Google</a></li> <li><a href="">Enough with microservices</a></li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>Introducing NoDB - a Pythonic Object Store for S3</strong></a></p> <ul> <li>Released in April 2017 by Rich Jones</li> <li>An incredibly simple, Pythonic object store based on Amazon's S3 static file storage.</li> <li>NoDB isn't a database.. but it sort of looks like one!</li> <li>Kind of like a document database, supports indexing</li> <li>Can use Pickling or JSON</li> <li>Mostly useful for <strong>prototyping</strong>, <strong>casual hacking</strong>, and (maybe) even low-traffic <strong>server-less databases</strong> for <a href=""><strong>Zappa</strong></a> <strong>apps</strong>!</li> <li>Can see a few use cases for <strong>NoDB</strong>: <ul> <li>Prototyping schemas</li> <li>Storing API event responses for later replay</li> <li>Capturing event logs</li> <li>Storing simple form data (email addresses, etc.)</li> <li>Storing non-relational analytics data</li> <li>Firing Lambda event triggers</li> <li>Version controlling evolving Python objects</li> <li>Storing and loading trained machine learning models</li> </ul></li> <li></li> </ul> <p><strong>Brian #3:</strong> <a href=""><strong>Elizabeth for mock data</strong></a> Part 1: Part 2: pytest plugin:</p> <p><strong>Michael #4:</strong> <a href=""><strong>What’s New In Python 3.7</strong></a></p> <ul> <li>Lang: More than 255 arguments can now be passed to a function, and a function can now have more than 255 parameters.</li> <li>Lang: <code>bytes.fromhex()</code> and <code>bytearray.fromhex()</code> now ignore all ASCII whitespace, not only spaces.</li> <li>Lang: Circular imports involving absolute imports with binding a submodule to a name are now supported.</li> <li>Module: <code>contextlib.asynccontextmanager()</code> has been added. <ul> <li>Similar to <code>contextmanager()</code>, but creates an asynchronous context manager.</li> <li>This function is a decorator that can be used to define a factory function for async with statement asynchronous context managers, without needing to create a class or separate <code>__aenter__()</code> and <code>__aexit__()</code> methods.</li> </ul></li> <li>Module:The dis() function now is able to disassemble nested code objects (the code of comprehensions, generator expressions and nested functions, and the code used for building nested classes).</li> <li>Module: math: New <code>remainder()</code> function, implementing the IEEE 754-style remainder operation.</li> <li>Optimization: Added two new opcodes: <code>LOAD_METHOD</code> and <code>CALL_METHOD</code> to avoid instantiation of bound method objects for method calls, which <strong>results in</strong> <strong>method calls being faster up to 20%</strong>.</li> <li>Optimization: The <code>os.fwalk()</code> function has been sped up by 2 times.</li> </ul> <p><strong>Brian #5: Hypothesis Testing</strong></p> <ul> <li><a href="">Unleash the Test Army</a></li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>Heroku switching default to v3.6.1</strong></a></p> <ul> <li>Effective Tuesday, June 20th, 2017, new Python applications pushed to Heroku will use the python-3.6.1 runtime by default (instead of python-2.7.13).</li> <li>Existing applications will not be affected by this change.</li> <li>“Lots of new projects start out on heroku all the time, so this is really great news for python 3 adoption.”</li> <li>“Python 3 is really happening. I was actually a little worried about the future of Python for a while.”</li> </ul>
Jun 15, 2017
#29 Responsive Bar Charts with Bokeh, Flask, and Python 3
<h1>Python Bytes 29</h1> <p>Brought to you by Rollbar! <a href=""></a></p> <p><strong>Brian #1:</strong> <a href=""><strong>Responsive Bar Charts with Bokeh, Flask and Python 3</strong></a></p> <ul> <li>by Matt Makai at</li> <li>A walkthrough example of putting together a flask app that uses Bokeh bar charts to visualize data.</li> <li>All steps included, no previous experience with Flask or Bokeh required.</li> <li>Nice explanation of what the code does without going into too much detail.</li> <li>Good jumping off point for further learning, but complete enough to be useful right away.</li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>Zappa</strong></a> <a href=""><strong>Serverless Python Web Services</strong></a></p> <ul> <li><strong>Zappa</strong> makes it super easy to build and deploy all Python WSGI applications on AWS Lambda + API Gateway</li> <li>Think of it as "serverless" web hosting for your Python apps. </li> <li>That means <strong>infinite scaling</strong>, <strong>zero downtime</strong>, <strong>zero maintenance</strong> - and at a fraction of the cost of your current deployments!</li> <li>Better still, with Zappa you only pay for the milliseconds of server time that you use, so it's many <strong>orders of magnitude cheaper</strong> than VPS/PaaS hosts and in most cases, it's completely free. Plus, there's no need to worry about load balancing or keeping servers online ever again.</li> <li>Asynchronous Task Execution: </li> </ul> <pre><code> from flask import Flask from zappa.async import task app = Flask(__name__) @task def make_pie(): """ This takes a long time! """ ingredients = get_ingredients() pie = bake(ingredients) deliver(pie) @app.route('/api/order/pie') def order_pie(): """ This returns immediately! """ make_pie() return "Your pie is being made!" </code></pre> <p><strong>Brian #3:</strong> <a href=""><strong>Using a local cache for pip packages</strong></a></p> <ul> <li>In <a href=""></a>, <strong>Local package store,</strong> we talked about using pip to cache pypi projects to allow offline installation: <ul> <li><code>$ pip download --cachedir &lt;somePackage&gt;</code></li> <li><code>$ pip install --no-index --find-links=/tmp/wheelhouse somePackage</code></li> </ul></li> <li>Well, Dominic does us one better by wrapping these commands in a couple of aliases. </li> <li>However, his version uses <code>pip install</code> <code>--``download</code>, which has been deprecated. Here’s a version with the new syntax: <ul> <li><code>alias pipcache='pip download --cache-dir ${HOME}/.pip-packages'</code></li> <li><code>alias pipinstall='pip install --no-index --find-links=file://${HOME}/.pip-packages/'</code></li> </ul></li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>Building game AI using ML: Working with TensorFlow, Keras, and the Intel MKL in Python</strong></a></p> <ul> <li>From the ActivePython guys</li> <li>a classic arcade space shooter game that features enemies powered by machine learning</li> <li>we decided to build a Neural Network to drive the behaviour of the enemies in the game</li> <li>For the game part of things, we’re using PyGame</li> <li>In the training mode, the enemies fire randomly, and then each shot taken by the enemy is recorded as a hit or a miss along with its initial relative position/velocity values. Every one of these shots becomes a row in the training matrix and the network is trained in “realtime” after every row is added so you can see the network build and develop as you train.</li> <li>LESSONS LEARNED <ul> <li>Choosing the right data to train your network is important.</li> <li>“Prepping” your data is key.</li> <li>Experiment with network topology.</li> <li>Visualization is important.</li> </ul></li> </ul> <p><strong>Brian #5:</strong> <a href=""><strong>Debug Test Failures With Pdb</strong></a></p> <ul> <li>by Raphael Pierzina</li> <li>Debugging code with pytest, using: <ul> <li>--pdb to jump into the debugger at the point of failure</li> <li>-x to stop after first failure</li> <li>--lf to re-run all the tests that failed last time</li> </ul></li> <li><strong>Note:</strong> Yes. All this and more is covered in <a href="">Python Testing with pytest</a>. <ul> <li>Shameless plug for my book. Raphael is one of the technical reviewers. </li> <li>Thank you, Raphael!</li> </ul></li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>Monitoring my VOIP provider with Home Assistant</strong></a></p> <ul> <li><strong>Integrating it into Home Assistant:</strong> Use <a href=""></a> as a home automation platform in my house. </li> <li>It’s written in Python, open source, and has a large community surrounding it. </li> <li>Unfortunately, there wasn’t anything already built for my Cisco ATA.</li> <li>Decided to write, an open source my first python module called <a href="">pyciscsospa</a> you can download it and use it for your own ATA as well.</li> <li>Receive a push notification on my phone when the phone lines go down and come back up</li> </ul>
Jun 08, 2017
#28 The meaning of _ in Python
<p><strong>Brian #1:</strong> <a href=""><strong> : PEP 8 — the Style Guide for Python Code</strong></a></p> <ul> <li>"This stylized presentation of the well-established PEP 8 was created by Kenneth Reitz (for humans)."</li> <li>From PEP 8: "This document gives coding conventions for the Python code comprising the standard library in the main Python distribution."</li> <li>PEP8 is not only used for the standard library. Many if not most open source Python packages adhere to at least most of the PEP8 recommendations</li> <li>testing plugins can help you make sure your code meets the guidelines (for good or bad).</li> <li>The presentation is easy to read, with a left side clickable table of contents.</li> <li>Nice color coded examples. Green for good, Red for bad.</li> <li>links to specific items make it easy to share with others something specific.</li> <li>Good advice, but don't be a pep8-bully.</li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>Tokio: Asyncio event loop written in Rust language</strong></a></p> <ul> <li>Asyncio event loop written in Rust language</li> <li>It is still in alpha stage. </li> <li>It provides most of asyncio event loop apis, except udp. </li> <li>TCP api is more or less stable</li> <li>Aiohttp tests pass with tokio loop (~1800 tests)</li> <li>Mostly interesting as an example of Rust + Python</li> <li>Project is still in early stage of development</li> </ul> <p><strong>Brian #3:</strong> <a href=""><strong>Python Boilerplate</strong></a></p> <ul> <li>Interactive online tool for creating script and small project boilerplate code.</li> <li>Just starting, with "how to help" link.</li> <li>Select <ul> <li>Python 2 or 3</li> <li>executable script or not</li> <li>argparse</li> <li>logging</li> <li>.gitignore</li> <li>Flask</li> <li>unittest or pytest</li> <li>tox</li> </ul></li> <li>fills in, plus other files like, requirements.txt, tox.ini, etc.</li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>Instagram switching to Python 3 on one branch</strong></a></p> <ul> <li>Ancient Django but still productive</li> <li>Ran out of 32-bit user IDs before they ran out of Django power. </li> <li>Added sharing support to Django Orem</li> <li>Turned off GC for perf</li> <li>Upgraded entirely to 3.6 in a few months</li> <li>Why? <ul> <li>Type hints</li> <li>Scaling server perf</li> <li>asyncio</li> </ul></li> <li>Python 3 is where the future community work is happening </li> <li>Strategies <ul> <li>No user impact</li> <li>Still shipping</li> <li>Testing process was interesting</li> </ul></li> <li>This is a concrete roadmap for every large company</li> </ul> <p><strong>Brian #5:</strong> <a href=""><strong>The Meaning of Underscores in Python</strong></a> </p> <ul> <li>single and double underscore meanings</li> <li>dunder is "double underscore" <ul> <li>Single Leading Underscore: <code>_var</code></li> </ul></li> <li>method or variable for internal use</li> <li>convention only</li> <li>doesn't apply to <code>collection.namedtuple</code> <ul> <li>Single Trailing Underscore: <code>var_</code></li> </ul></li> <li>used to avoid name collision with keywords <ul> <li>Double Leading Underscore: <code>__var</code></li> </ul></li> <li>internal use by a single class level.</li> <li>Python will name mangle this so that subclasses don't have to avoid parent class double leading underscore names <ul> <li>Double Leading and Trailing Underscore: <code>__var__</code></li> </ul></li> <li>no name mangling</li> <li>special names. dunder methods</li> <li><code>__call__</code> and <code>__init__</code>, etc. <ul> <li>Single Underscore: <code>_</code></li> </ul></li> <li>in code : temp variable, don't care variable</li> <li>won't get a warning if you don't reference it again</li> <li>in REPL: last value</li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>The future is looking bright for Python</strong></a></p> <ul> <li>Stack Overflow recently released a cool new tool called <a href="">Trends</a> (previously covered)</li> <li>Check out the <a href=";utm_medium=blog&amp;utm_campaign=trends&amp;utm_content=blog-link&amp;">Most Popular Languages</a> trend chart</li> <li>Python has, by a very large margin, the greatest positive slope (future?)</li> <li>And <a href="">Py3 vs Py2</a></li> </ul>
Jun 02, 2017
#27 The PyCon 2017 recap and functional Python
<ul> <li>All videos available: <a href=""></a></li> <li>Lessons learned: <ul> <li>pick up swag on day one. vendors run out.</li> <li>take business cards with you and keep them on you</li> <li>Not your actual business cards unless you are representing your company.</li> <li>Cards that have your social media, github account, blog, or podcast or whatever on them.</li> <li>3x3 stickers are too big. 2x2 plenty big enough</li> <li>lightening talks are awesome, because they are a lot of ranges of speaking experience</li> <li>will definitely do that again</li> <li>try to go to the talks that are important to you, but don’t over stress about it, since they are taped. However, it would be lame if all the rooms were empty, so don’t everybody ditch.</li> <li>lastly: everyone knows Michael. </li> </ul></li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>How to Create Your First Python 3.6 AWS Lambda Function</strong></a></p> <ul> <li>Tutorial from <a href="">Full Stack Python</a></li> <li>Walks you through creating an account</li> <li>Select your Python version (3.6, yes!)</li> <li><code>def lambda_handler(event, context): …</code> # write this function, done!</li> <li>Set and read environment variables (could be connection strings and API keys)</li> </ul> <p><strong>Brian #3:</strong> <a href=""><strong>How to Publish Your Package on PYPI</strong></a></p> <ul> <li>jetbrains article <ul> <li>structure of the package</li> <li>oops. doesn't include src, see</li> <li>decent discussion of a the contents of the file (but interestingly absent is an example file)</li> <li>good discussion of .pypirc file and links to the test and production PyPi</li> <li>example of using twine to push to PyPI</li> <li>overall: good discussion, but you'll still need a decent example.</li> </ul></li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>Coconut: Simple, elegant, Pythonic functional programming</strong></a></p> <ul> <li>Coconut is a functional programming language that compiles to Python. </li> <li>Since all valid Python is valid Coconut, using Coconut will only extend and enhance what you're already capable of in Python.</li> <li><code>pip install coconut</code> <ol> <li>Some of Coconut’s major features include built-in, syntactic support for:</li> <li>Pattern-matching,</li> <li>Algebraic data-types,</li> <li>Tail call optimization,</li> <li>Partial application,</li> <li>Better lambdas,</li> <li>Parallelization primitives, and</li> <li>A whole lot more, all of which can be found in <a href="">Coconut’s detailed documentation</a>.</li> </ol></li> <li>Talk Python episode coming in a week</li> </ul> <p><strong>Brian #5:</strong> <a href=""><strong>Choose a licence</strong></a></p> <ul> <li>MIT : simple and permissive</li> <li>Apache 2.0 : something extra about patents.</li> <li>GPL v3 : this is the contagious one that requires derivitive work to also be GPL v3</li> <li>Nice list with overviews of what they all mean with color coded bullet points: <a href=""></a></li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>Python for Scientists and Engineers</strong></a></p> <ul> <li><strong>Table of contents</strong>:</li> <li><strong>Beginners Start Here:</strong> <ul> <li><a href=""><strong>Create a Word Counter in Python</strong></a></li> <li><a href=""><strong>An introduction to Numpy and Matplotlib</strong></a></li> <li><a href=""><strong>Introduction to Pandas with Practical Examples (New)</strong></a></li> </ul></li> <li><strong>Main Book</strong> <ul> <li><a href=""><strong>Image and Video Processing in Python</strong></a></li> <li><a href=""><strong>Data Analysis with Pandas</strong></a></li> <li><a href=""><strong>Audio and Digital Signal Processing (DSP)</strong></a></li> <li><a href=""><strong>Control Your Raspberry Pi From Your Phone / Tablet</strong></a></li> </ul></li> <li><strong>Machine Learning Section</strong> <ul> <li><a href=""><strong>Machine Learning with an Amazon like Recommendation Engine</strong></a></li> <li><a href=""><strong>Machine Learning For Complete Beginners:</strong></a> <em><em></em></em>Learn how to predict how many Titanic survivors using machine learning. No previous knowledge needed!</li> <li><a href=""><strong>Cross Validation and Model Selection</strong></a>: In which we look at cross validation, and how to choose between different machine learning algorithms. Working with the Iris flower dataset and the Pima diabetes dataset.</li> </ul></li> <li><strong>Natural Language Processing</strong> <ul> <li><a href=""><strong>Introduction to NLP and Sentiment Analysis</strong></a></li> <li><a href=""><strong>Natural Language Processing with NTLK</strong></a></li> <li><a href=""><strong>Intro to NTLK, Part 2</strong></a></li> <li><a href=""><strong>Build a sentiment analysis program</strong></a></li> <li><a href=""><strong>Sentiment Analysis with Twitter</strong></a></li> <li><a href=""><strong>Analysing the Enron Email Corpus</strong></a>: The Enron Email corpus has half a million files spread over 2.5 GB. When looking at data this size, the question is, where do you even start?</li> <li><a href=""><strong>Build a Spam Filter using the Enron Corpus</strong></a></li> </ul></li> </ul> <p><strong>In other news</strong>:</p> <ul> <li><a href="">Python Testing with pytest</a> Beta release and initial feedback is going very well.</li> </ul>
May 25, 2017
#26 How have you automated your life, or CLI, with Python?
<p>Sponsored by rollbar: <a href=""></a></p> <p><strong>Brian #1: Two part series on interactive terminal applications</strong></p> <p><strong>Part 1:</strong> <a href=""><strong>4 terminal applications with great command-line UIs</strong></a></p> <ul> <li>For Comparison: both ok but could be better <ul> <li>MySQL REPL</li> <li>Python REPL</li> </ul></li> <li><a href="">bpython</a> adds autocompletion and other goodies <ul> <li>also check out <a href="">ptpython</a> as a REPL replacement</li> </ul></li> <li><a href="">mycli</a> adds context aware completion to MySQL <a href="">mycli</a> - <a href="">pgcli</a> for postgress that adds fuzzy search</li> <li><a href="">fish</a> : like bash, but has better search history</li> </ul> <p><strong>Part 2:</strong> <a href=""><strong>4 Python libraries for building great cli's</strong></a></p> <ul> <li><a href="">prompt_toolkit</a> - for building a REPL like interface <ul> <li>includes command history, auto-suggestion, auto-completion</li> </ul></li> <li><a href="">click</a> <ul> <li>includes pager and ability to launch an editor</li> </ul></li> <li><a href="">fuzzyfinder</a> - make suggestions <ul> <li>article shows how to combine that with prompt_toolkit</li> </ul></li> <li><a href="">pygments</a> - syntax highlighting</li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>How have you automated your life with python?</strong></a></p> <ul> <li>There is something magical about writing code that interacts with the physical world.</li> <li>I have a script which runs every 5 minutes between 17:00 and 17:30 which scrapes the train times website and sends me desktop notifications saying whether or not my trains home are delayed / cancelled.</li> <li>I recently wrote a quick python script that tells me when my girlfriend comes home: It sniffs the network for DHCP traffic, when her phone joins the wifi network outside it uses the say command to let me know.</li> <li>Wrote a script to check if nearby ice cream shops are stocking my favourite (rare) flavour by scanning their menu page for keywords.</li> <li>A script to check the drive time too/from work using a route with tolls or without tolls.. to try and save some money when the times aren't too different. Using google maps API and a flask site.</li> <li>I have a script that generates weekly status update emails based off my git commit messages and pull requests. It also creates timesheets in Harvest based on the projects I'm assigned.</li> <li>I have thrown together some python that automatically controls my reverse-cycle AC system so that it makes optimal use of my solar panels on my roof.</li> </ul> <p><strong>Brian #3</strong>: <a href=""><strong>Building a Simple Birthday App with Flask-SQLAlchemy</strong></a></p> <ul> <li>Nice simple application with a clear need. <ul> <li>Keep track of upcoming birthdays</li> <li>Avoid Faceboook</li> <li>Build a simple Flask app</li> <li>Try SQLAlchemy</li> </ul></li> </ul> <p><strong>Sponsored by Rollbar</strong>, try them at <a href=""></a> and don't forget to visit their booth at PyCon!</p> <p><strong>Michael #4:</strong> <a href=""><strong>Spelling with Elemental Symbols</strong></a></p> <ul> <li>How does it work? <ul> <li>Input: "Amputations"</li> <li>Output: "AmPuTaTiONS", "AmPUTaTiONS"</li> </ul></li> <li>Generating Character Groupings: <ul> <li>'AmPuTaTiONS' <code>(2,2,2,2,1,1,1)</code></li> <li>'AmPUTaTiONS' <code>(2,1,1,2,2,1,1,1)</code></li> <li>How many are there in general for a given word? <code>fib(n + 1)</code>!</li> </ul></li> <li>Addressing Performance Issues: A few attempts don’t add much but</li> <li>Memoization: The technique of saving a function's output and returning it if the function is called again with the same inputs. A memoized function only needs to generate output once for a given input. This can be very helpful with expensive functions that are called many times with the same few inputs, but only works for pure functions. → 30% faster</li> <li>Algorithms: Switch to directed graphs and recursion, changes O(2^n) to O(n) and time from 16min to 10 sec.</li> <li>Learned a great deal along the way. This project introduced: <ul> <li>Combinatorics</li> <li>Performance profiling</li> <li>Time complexity</li> <li>Memoization</li> <li>Recursion</li> <li>Graphs and trees</li> </ul></li> </ul> <p><strong>Brian #5:</strong> <strong>IDE's for beginners</strong></p> <ul> <li><a href="">Recent discussion on Reddit about Thonny</a></li> <li>I have mixed feelings about encouraging beginner IDE's. <ul> <li>Mostly negative feelings.</li> <li>And yet there is IDLE, there is Thonny, ...</li> </ul></li> <li>Are these useful? Anti-useful?</li> <li>Isn't learning a decent editor part of learning to program?</li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>PDF Plumber</strong></a></p> <ul> <li>Plumb a PDF for detailed information about each char, rectangle, line, et cetera — and easily extract text and tables.</li> <li>Visual debugging with <code>.to_image()</code></li> <li>Extracting tables <ul> <li>pdfplumber's approach to table detection borrows heavily from Anssi Nurminen's master's thesis, and is inspired by Tabula. It works like this:</li> <li>For any given PDF page, find the lines that are (a) explicitly defined and/or (b) implied by the alignment of words on the page.</li> <li>Merge overlapping, or nearly-overlapping, lines.</li> <li>Find the intersections of all those lines.</li> <li>Find the most granular set of rectangles (i.e., cells) that use these intersections as their vertices.</li> <li>Group contiguous cells into tables.</li> <li>Check out the demonstrations section.</li> </ul></li> </ul>
May 19, 2017
#25 Could we have more in-database machine learning please?
<p><strong>Michael #1:</strong> <a href=""><strong>Python in SQL Server 2017: enhanced in-database machine learning</strong></a></p> <ul> <li>in-database analytics and machine learning with Python in SQL Server.</li> <li>Why? <ul> <li><strong>Elimination of data movement:</strong> You no longer need to move data from the database to your Python application or model. Instead, you can build Python applications in the database. </li> <li><strong>Easy deployment:</strong> Once you have the Python model ready, deploying it in production is now as easy as embedding it in a T-SQL script</li> <li><strong>Rich extensibility:</strong> You can install and run any of the latest open source Python packages in SQL Server to build deep learning and AI applications on huge amounts of data in SQL Server.</li> <li>Works in express edition of SQL Server</li> </ul></li> <li>The standard open source CPython interpreter (version 3.5) and some Python packages commonly used for data science are downloaded and installed during SQL Server setup if you choose the Python option in the feature tree.</li> <li>Currently, a subset of packages from the popular Anaconda distribution is included along with Microsoft’s RevoScalePy package. The set of packages available for download will evolve as we move toward general availability of this feature.</li> </ul> <p><strong>Brian #2</strong>: <a href=""><strong>Generating Fake Data for Python Unit Tests with Faker</strong></a></p> <ul> <li>Used in <a href="">Content Negotiation with the Pyramid Web Framework</a></li> <li><a href="">Faker</a> is great. From the pypi page: <ul> <li>“<em>Faker</em> is a Python package that generates fake data for you. Whether you need to bootstrap your database, create good-looking XML documents, fill-in your persistence to stress test it, or anonymize data taken from a production service, Faker is for you.”</li> </ul></li> <li>The article above is a great introduction to show you what you can do with it.</li> <li>Faker includes localization, so you can get the output to match the proper language you want.</li> </ul> <p><strong>Brian #3:</strong> <a href=""><strong>Stack Overflow Trends tool</strong></a></p> <ul> <li>Stack overflow introduces a trends tool starting with a graph that shows the questions using the terms Python, PHP, and Perl, plotted year vs % of total questions. <ul> <li><a href=""></a></li> </ul></li> <li>Python’s growth since 2012 has been fairly steady, growing from ~3.5% to over 8%</li> <li>Plus, the trends tool looks fun.</li> </ul> <p><strong>Sponsorship slots are available right now. Please contact us if you’re interested.</strong></p> <p><strong>Michael #4:</strong> <a href=""><strong>We asked 20,000 people who they are and how they’re learning to code</strong></a></p> <ul> <li>Thanks Alan Jones</li> <li><strong>Who participated? More than 20,000 new coders responded to this survey. These are people who have been coding for less than 5 years.</strong> <ul> <li>62% of them live outside the US</li> <li>their average age is 28 years old</li> <li>19% are women</li> <li>They’ve been coding for an average of 21 months</li> <li>25% have already landed their first developer job</li> </ul></li> <li>40% of them would like to either freelance or start their own business.</li> <li>Most of them are interested in working as web developers, but are also interested in a wide variety of developer specializations.</li> <li>Most new coders haven’t yet started listening to podcasts, but the ones who do listen to a wide range of them.</li> </ul> <p><strong>Brian #5</strong> <a href=""><strong>Introduction to Anomaly Detection</strong></a></p> <ul> <li><strong>Anomaly Detection Techniques</strong> <ul> <li>Simple Statistical Methods</li> <li>Challenges</li> </ul></li> <li><strong>Machine Learning-Based Approaches</strong> <ul> <li>Density-Based Anomaly Detection</li> <li>Clustering-Based Anomaly Detection</li> <li>Support Vector Machine-Based Anomaly Detection</li> <li>Building a Simple Detection Solution Using a Low-Pass Filter</li> <li>Moving Average Using Discrete Linear Convolution</li> </ul></li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>Beeware: A request for your help</strong></a></p> <ul> <li>What is Beeware? Started 4 years ago. Since then, the BeeWare project has grown to encompass support for mobile platforms, two alternate Python implementations, and a cross platform widget set - as well as the developer tools that started the project originally.</li> <li>Lots of work done over the last 6 months: <ul> <li>Extensive improvements to Batavia and VOC;</li> <li>An Android backend for Toga;</li> <li>A Django backend for Toga, enabling Toga apps to be deployed as web apps;</li> <li>A Winforms backend for Toga, enabling Toga apps to run on Windows with a modern appearance;</li> </ul></li> <li>Unfortunately, my contract with Jambon is coming to a close - which means my contributions to BeeWare will go back to being what my spare time allows.</li> <li>So - this is an appeal to you - the Python community. If you are excited by the prospect of having access to Python on mobile platforms, or you would like to write applications in Python that have completely native user interfaces - <strong>I need your help</strong> for just <strong>US$10 a month</strong> - you can <a href="">join the BeeWare project as a member</a>,</li> </ul> <p><strong>Other news</strong></p> <p><a href=""></a> is a regional PyCon for the Pacific Northwest.</p>
May 12, 2017
#24 I have a local PyPI server and so do you!
<p>Welcome to Python bytes. Episode 24, recorded May 2, 2017. This episode is sponsored by rollbar: <a href=""></a></p> <p><strong>Brian #1: <a href="">Learning Python Series by Doug Farrell</a>, published on</strong></p> <ul> <li>Doug Ferrell wrote for <a href="">Robotics and Beyond</a> , a STEM educational space</li> <li>This is part 1: <strong>Let’s Program with Python: Statements, Variables, and Loops</strong></li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>Geeking out in your older years</strong></a></p> <ul> <li>Over 500 respondents on <a href=""></a></li> <li>Age group: 60-85</li> <li>Opportunities: Help older adults become software dev proficient to <ul> <li>Connect with their grand children</li> <li>Fill the growing tech / teacher gap</li> <li>Mentor others</li> <li>Keep mentally active</li> <li>Pursue their hobbies</li> </ul></li> <li>Used MOOCs, online courses, books, and more to learn</li> <li>Felt somewhat isolated and disconnected <ul> <li>How could we all help? Or help them help themselves as a group?</li> </ul></li> </ul> <p><strong>Brian #3:</strong> <strong>Local package store</strong></p> <ul> <li>Not a story, just something I used for the plane trip that didn’t know worked before.</li> <li>Stops my quest for a easy to use local pypi server.</li> <li><p>Download without installing</p> <p><code>$ cd /tmp/wheelhouse</code> <code>$ python3.6 -m pip download &lt;somePackage&gt;</code> </p></li> <li><p>Then, later, probably within a virtual env</p> <p><code>$ python3.6 -m pip install --no-index --find-links=/tmp/wheelhouse somePackage</code> </p></li> <li><p><strong>Does it work with requirements files? Yes!</strong></p></li> </ul> <p><code>pip3 download -r /full_path_to/requirements.txt</code></p> <p><strong>Sponsored by rollbar</strong> <a href=""></a></p> <ul> <li><strong>Adding the Rollbar Python SDK is as easy as <code>pip install rollbar</code>.</strong></li> <li><strong>Make sure to check out Rollbar at Pycon. Grab swag and get a product demo.</strong> </li> <li>Visit <a href=""></a> and get the get the Bootstrap Plan free for free <ul> <li>100,000 rollbar events / mo, 180 days retention</li> </ul></li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>Modifying the Python language in 6 minutes</strong></a></p> <ul> <li>Anthony Shaw</li> <li>I’m writing my findings on how CPython works and show you how easy it is to modify the Python syntax.</li> <li>I’m going to show you how to add a new <strong>feature</strong> to the Python syntax.</li> <li><strong>Level 1: PEPs</strong></li> <li><strong>Level 2: Grammar</strong></li> <li>The <a href="">Grammar</a> file is simple text file describing all the elements of the Python language. This is used by not just CPython, but other implementations like PyPy to keep consistency and agree on the types of language semantics. <ul> <li>New statements <ul> <li><code>incr_stmt: '++'</code></li> <li><code>decr_stmt: '--'</code></li> </ul></li> </ul></li> <li><strong>Level 3 : Lexer:</strong> There are four steps that Python takes when you hit return: lexing, parsing, compiling, and interpreting. Lexing is breaking the line of code you just typed into tokens.</li> <li><strong>Level 4 : Parser</strong>: The parser takes those tokens and generates a structure that shows their relationship to each other. For Python and many other languages, this is the Abstract Syntax Tree (or AST). </li> <li><strong>Level 5: Compiler:</strong> The compiler then takes the syntax tree and ‘visits’ each branch, the CPython compiler has a method for visiting a statement, called <code>compile_visit_stmt</code> which is just a big switch statement looking at the statement kind.</li> <li><a href=""></a></li> </ul> <p><strong>Brian #5:</strong> <a href=""><strong>colorful</strong></a></p> <ul> <li>“Terminal string styling done right, in Python”</li> <li>Add color to terminal applications with a pretty easy to read syntax</li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>Five steps to add the _bling_ factor your Python package</strong></a> <em>*</em>*</p> <ul> <li>by Tjelvar Olsson</li> <li>Step 1: Host the documentation on readthedocs</li> <li>Step 2: Set up continuous integration testing on Travis Ci</li> <li>Step 3: Calculate your code coverage using Codecov</li> <li>Step 4: Upload your Package to PyPi</li> <li>Step 5: Add badges to your project’s README file</li> </ul>
May 03, 2017
#23 Can you grok the GIL?
<p><strong>Sponsored by ADVANCE DIGITAL. Find your Python web job at <a href=""></a></strong></p> <p><strong>Brian #1:</strong> <a href=""><strong>Grok the GIL</strong> - <strong>How to write fast and thread-safe Python</strong></a> </p> <ul> <li>A. Jesse Jiryu Davis teaches us about the GIL, and how to use that knowledge to decide between threads and processes for parallelism.</li> <li>From the article: <ul> <li>The GIL's effect on the threads in your program is simple enough that you can write the principle on the back of your hand: "One thread runs Python, while N others sleep or await I/O."</li> </ul></li> <li>Discusses and Cooperative multitasking and Preemptive multitasking</li> <li>When can a Python process be interrupted? (between bytecodes)</li> <li>When do you need to use thread protection? (less than you think)</li> <li><em>A. Jesse Jiryu Davis will be speaking at</em> <a href=""><em>PyCon 2017</em></a><em>, which will be held May 17-25 in Portland, Oregon. Catch his talk,</em> <a href=""><em>Grok the GIL: Write Fast and Thread-Safe Python</em></a><em>, on Friday, May 19.</em></li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>The New NBA by Mark Cuban</strong></a></p> <ul> <li>Introduction to machine learning in Python &amp; Jupyter notebooks</li> <li>Mark Cuban using Python and ML to play with his NBA team</li> <li>“We have a team at the Mavs but I need to know it to help define strategy and make decisions”</li> </ul> <p><strong>Brian #3:</strong> <a href=""><strong>Ian Cordasco gets a Community Service Award from PSF</strong></a></p> <ul> <li>Ian was on <a href="">Test &amp; Code, episode 13</a>, talking about Betamax</li> <li>From the announcement: <ul> <li>Ian Cordasco has been the PSF’s Election Administrator since 2015, volunteering his efforts for this important role. </li> <li>Cordasco frequently mentors newer coders and supports their Python endeavors. </li> <li>The Python Software Foundation award the 2017 Q1 Community Service Award to Ian Cordasco for his contributions to PSF elections and active mentoring of women in Python community.</li> <li>Cordasco has a history of going out of his way to support and encourage female developers. When <a href="">Carol Willing</a>, a developer for the Jupyter project, wanted to work on the Requests library, she got in touch with Cordasco. “We worked together on the project and my first commit to the Requests library got accepted!” Cordasco later wrote a fantastic post about it on his <a href="">blog</a>.</li> </ul></li> </ul> <p><strong>Sponsored by ADVANCE DIGITAL</strong></p> <ul> <li>They have a small team of developers who work in an agile/devops environment– you will make an impact with your work quickly</li> <li>They are mostly a python shop, but there is an opportunity to introduce and run other technologies at scale</li> <li>They fund employee development and conference attendance</li> <li>They are located in beautiful Jersey City, one stop from Manhattan on the PATH</li> <li>They are one of the 10 largest news sites by traffic in the US</li> <li>Apply at <strong><a href=""></a></strong></li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>Release of IPython 6.0</strong></a></p> <ul> <li>by Matthias Bussonnier</li> <li>IPython goes Python 3 only</li> <li>Our personal experience writing Python3-only source code. <ul> <li>The size of the IPython codebase has decreased by about 1500 lines of Python code relative to the last release. Of course, that’s not solely due to the removal of Python 2 support, but a non-negligible amount is.</li> <li>even more remarkable in light of completely new features that required adding hundreds of lines of code</li> <li>This change eases the burden on contributors to IPython. Contributors can can spend less time thinking “what about Python 2”, or rewriting a pull request because the Python 2 test suite fails. </li> <li>At the same time, our tests now complete more quickly on continuous integration services because they need to run on fewer versions of Python.</li> <li>From a developer point of view we are extremely pleased with having the possibility to write Python3-only code, and are looking forward to even more improvements like <a href="">pathlib</a>.</li> <li>We hope you will enjoy this release. It will be the base for some awesome features, like async/await REPL.</li> </ul></li> </ul> <p><strong>Brian #5:</strong> <a href=""><strong>Testing &amp; Packaging</strong></a></p> <ul> <li>Hynek Schlawack describes why he was convinced to use a src directory in package distributions he works on.</li> <li>Just use a src dir (of course it doesn’t have to be exactly “src” but that’s the convention and it should be different than your package name), it will make your life easier.</li> <li>Without it, it is easy to think you are testing an installed package, but you’re really testing the modules before install. These can be different. Better to test as close to how your users will see the package as possible, and using a src dir helps that.</li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>AWS Lambda adds Python 3.6 support</strong></a></p> <ul> <li>AWS Lambda is a compute service that lets you run code without provisioning or managing servers. </li> <li>AWS Lambda executes your code only when needed and scales automatically, from a few requests per day to thousands per second. </li> <li>You pay only for the compute time you consume - there is no charge when your code is not running.</li> <li>You can also build <a href="">serverless</a> applications composed of functions that are triggered by events and automatically deploy them using AWS CodePipeline and AWS CodeBuild. </li> <li>Already good things cometh: <a href="">Zappa has Python 3 support</a>.</li> </ul>
Apr 26, 2017
#22 PYTHONPATH considered harmful
<p><strong>Sponsored by <a href="">ADVANCE DIGITAL</a></strong>. Find your rewarding Python job at <a href=""></a></p> <p><strong>#1 Brian:</strong> <a href=""><strong>PYTHONPATH considered harmful</strong></a></p> <ul> <li>Don’t do it.</li> <li>You might not regret it today. But later you will.</li> <li>Mucks up distribution searches, etc. <ul> <li>“For one, most directories are poorly suited to be on the Python search path. Consider, for example, the root directory of a typical Python project: it contains <code></code> -- and so, if it were added to the current search path, <code>import setup</code> would become possible. (This is one reason to have <code>src/</code> directories.) Often, directories added unwisely to the Python search path cause files to be imported from paths they do not expect to, and surprisingly conflict.”</li> </ul></li> </ul> <p><strong>#2 Michael:</strong> <a href="">keon/algorithms</a></p> <ul> <li>Minimal examples of data structures and algorithms in Python</li> <li>Topics include <ul> <li>Array</li> <li>circular_counter</li> <li>flatten</li> <li>garage</li> <li>merge_intervals</li> <li>graphs</li> <li>clone_graph</li> <li>find_path</li> <li>traversal</li> <li>trees</li> <li>etc.</li> </ul></li> </ul> <p><strong>#3 Brian:</strong> <strong><a href="">Glyph on attrs</a></strong></p> <ul> <li>We talked about <code>attrs</code> in <a href="">episode 11</a>, and pointed to the project and the docs.</li> <li>I came across good article introducing why you should use <code>attrs</code>, by glyph, from 2016.</li> <li>The one Python library everyone needs: <a href=""></a></li> <li>Discusses <ul> <li>problems with using lists and tuples as data structures.</li> <li>creating your own classes properly. </li> <li>possible problems with <code>namedtuple</code> (-ish. I still love <code>namedtuple</code>).</li> </ul></li> </ul> <p><strong>Sponsored by <a href="">ADVANCE DIGITAL</a></strong></p> <ul> <li>A small team of developers who work in an agile/devops environment– you will make an impact with your work quickly</li> <li>Are mostly a python shop, but there is an opportunity to introduce and run other technologies at scale</li> <li>Fund employee development and conference attendance</li> <li>Are located in beautiful Jersey City, one stop from Manhattan on the PATH</li> <li>Are one of the 10 largest news sites by traffic in the US</li> <li>Apply at <a href=""></a></li> </ul> <p><strong>#4 Michael:</strong> <a href=""><strong>Curio for Python 3.5+ concurrency</strong></a></p> <ul> <li>Curio is a library for performing concurrent I/O and common system programming tasks such as launching subprocesses and farming work out to thread and process pools. </li> <li>Curio is solely concerned with the execution of coroutines. A coroutine is a function defined using async def.</li> <li>It uses Python coroutines and the explicit <code>async</code>/<code>await</code> syntax introduced in Python 3.5. </li> <li>Its programming model is based on cooperative multitasking and existing programming abstractions such as threads, sockets, files, subprocesses, locks, and queues. </li> <li>All sorts of cool constructs: <code>AsyncThreads</code>, <code>UniversalQueues</code>, async file I/O, <code>Tasks</code>, and more.</li> </ul> <p><strong>#5 Brian:</strong> <strong>Python Package src-ery</strong></p> <ul> <li>"Use the src, Luke"</li> <li>"To src, or not to src, that is the question"</li> </ul> <p>Answering a listener question about Python packaging. In episode 15: Digging into Python Packaging, we mentioned to articles about getting started with packaging. In the comments, Kristof Claes noted that these references were in conflict with a couple of other references:</p> <ul> <li>pytest “Good Integration Practices”, <a href=""></a></li> <li>ionel’s “Packaging a Python library”, <a href=""></a></li> </ul> <p>Both of these strongly encourage the use of a “src” directory when setting up a package for distribution. There seems to be good reasons to use “src”. Many of the reasons are around the idea that during testing, you should be testing an installed version of the code. I have no reason to disagree with Ionel’s arguments and the pytest documentation recommendation.</p> <p>However:</p> <ul> <li>The pypa doesn’t bring this up when discussing distribution: <ul> <li><a href=""></a></li> </ul></li> <li>The pypa sample project doesn’t use “src”: <ul> <li><a href=""></a></li> </ul></li> <li>Many popular packages don’t: <ul> <li>requests: <a href=""></a></li> <li>pytest itself: <a href=""></a></li> </ul></li> </ul> <p>Why not?</p> <ul> <li>The pytest recommendation is subtle. It recommends using “src” if you need to include a dunder init file in the tests directory. Otherwise, the local code will be tested instead of the installed code, in part to test the installation and to test a library from the perspective of a user.</li> <li>pytest also recommends against having a top level dunder init in the tests directory. And this is a stronger recommendation.</li> <li>But Ionel’s points are not just around the use of pytest.</li> <li>So this is still really an open question to the Python community. <ul> <li>If it’s great to use “src” instead of top level packages, why aren’t more projects doing this?</li> <li>Why doesn’t the PyPA mention it?</li> </ul></li> </ul> <p><strong># 6 Michael:</strong> <a href=""><strong>Intel Pulls Funding from OpenStack Effort It Founded With Rackspace</strong></a></p> <ul> <li>Intel and Rackspace were collaborating on a project called OpenStack Innovation Center</li> <li>Launched in July 2015. </li> <li>A source close to the effort said initial funding was supposed to last through 2018, but Intel pulled it early.</li> <li>A Rackspace spokeswoman said “OSIC’s objective was to create the world’s largest OpenStack developer cloud and develop enterprise capabilities within OpenStack. It quickly accomplished the first goal, and has made great progress toward the second.”</li> <li>Some 30 Rackspace employees who had been working at the innovation center have been given two weeks to find new jobs at the San Antonio-based company.</li> <li>Story here is we all need to think about funding projects and diversification.</li> </ul> <p><strong>Our news:</strong></p> <p><strong>Michael</strong>: Hurry up and register for EuroPython: <a href=""></a> Earlybird sold out.</p>
Apr 18, 2017
#21 Python has a new star framework for RESTful APIs
<p>This episode has been <strong>sponsored by Rollbar</strong>. Get a special offer via <a href=""></a></p> <p><strong>#1 Brian:</strong> <a href=""><strong>profile and pstats — Performance Analysis</strong></a></p> <ul> <li>Doug Hellman is working on the Python 3 MOTW series that was so successful for Python 2.</li> <li>Recent edition is profile and pstats, for profiling parts of your code you may have concerns with and finding out where the slow bits are.</li> </ul> <p><strong>#2 Michael:</strong> <a href=""><strong>API Star by Tom Christie</strong></a></p> <ul> <li>A smart Web API framework, designed for Python 3.</li> <li>A few things to try right away:</li> </ul> <pre><code> $ pip3 install apistar $ apistar new --template minimal $ apistar run $ apistar test </code></pre> <ul> <li>API Star allows you to dynamically inject various information about the incoming request into your views using type annotation. <ul> <li>e.g. </li> </ul></li> </ul> <pre><code> def show_query_params(query_params: http.QueryParams): return { 'params': dict(query_params) } </code></pre> <ul> <li>You can instead set the status code or headers by annotating the view as returning a Response</li> </ul> <pre><code>def create_project() -&gt; Response: ... </code></pre> <ul> <li>Parameters are automatically passed into views from routes (annotations!):</li> </ul> <pre><code> def echo_username(user_id: int): return {'message': f'Welcome, user {user_id}!'} </code></pre> <ul> <li>Performance: Faster than sanic!</li> </ul> <p><strong>#3 Brian:</strong> <a href=""><strong>Yes, Python is Slow, and I Don’t Care</strong></a></p> <ul> <li>Optimize for your most expensive resource. That’s <strong>YOU</strong>, not the computer.</li> <li>Choose a language/framework/architecture that helps you develop quickly (such as Python). Do not choose technologies simply because they are fast.</li> <li>When you do have performance issues: find your bottleneck</li> <li>Your bottleneck is most likely not CPU or Python itself.</li> <li>If Python <strong>is</strong> your bottleneck (you’ve already optimized algorithms/etc.), then move the hot-spot to Cython/C</li> <li>Go back to enjoying getting things done quickly</li> </ul> <p><strong>#4 Michael:</strong> <a href=""><strong>A Quick Introduction: Hashing</strong></a></p> <ul> <li>Article by Gerald Nash</li> <li>Hashing is a method of determining the equivalence of two chunks of data. </li> <li>A cryptographic hash function is an irreversible function that generates a unique string for any set of data.</li> <li>Example</li> </ul> <pre><code> import hashlib as hash sha = hash.sha256() # Insert the string we want to hash sha.update('Hello World!') # Print the hexadecimal format of the binary hash we just created print(sha.hexdigest()) # 4d3cf15aa67c88742e63918825f3c80f203f2bd59f399c81be4705a095c9fa0e </code></pre> <ul> <li>Know when to choose “weak” hashes vs. strong ones</li> <li>Straight hashes are not enough for security (e.g. passwords). Use passlib and be done.</li> </ul> <p><strong>#5 Brian:</strong> <a href=""><strong>Wedding at Scale: How I Used Twilio, Python and Google to Automate My Wedding</strong></a></p> <ul> <li>gspread to access a google spreadsheet of guests and phone numbers</li> <li>SMS guests with twilio</li> <li>replies handled by a flask app</li> <li>gathered accept/decline/didn't reply statistics</li> <li>reminder texts</li> <li>food selections and replies and reminders, all handled by Python</li> </ul> <p><strong># 6 Michael:</strong> <a href=""><strong>python-alexa: A Python framework for Alexa Development</strong></a></p> <ul> <li>by Neil Stewart</li> <li>Ordered an amazon assistant.</li> <li>Before it arrived, I had challenged myself to develop something for it</li> <li>Project: VoiceOps, interact with an AWS account, such as telling me how many running and stopped instances there is or what RDS databases are in an account</li> <li>Wanted a framework that would make Alexa development super easy.</li> <li>Decided a new framework was needed: python-alexa</li> <li><a href="">python-alexa on github</a></li> <li><a href="">echo shim</a> for testing without hardware</li> </ul> <p><strong>Our news:</strong></p> <p><strong>Michael</strong>: Just added <a href="">full text search</a> (including within videos) to Talk Python courses.</p> <p><strong>Brian:</strong> <a href="">Netflix chaos engineering interview on Test &amp; Code</a></p>
Apr 13, 2017
#20 Finding similar but not identical images in 128 bits via Python
<p><strong>Sponsored by Rollbar, thank you! <a href=""></a></strong></p> <p><strong>#1 Brian:</strong> <a href=""><strong>Duplicate image detection with perceptual hashing in Python</strong></a></p> <ul> <li>Ben Hoyt</li> <li>From <a href=""></a>, Invitation-Only Travel Community</li> <li>We use a perceptual image hash called dHash (“difference hash”), which was developed by Neal Krawetz in his work on photo forensics. It’s a very simple but surprisingly effective algorithm that involves the following steps (to produce a 128-bit hash value) <ul> <li>Convert the image to grayscale</li> <li>Downsize to a 9x9 square of gray values (or 17x17 for a larger, 512-bit hash)</li> <li>Calculate the “row hash”: for each row, move from left to right, and output a 1 bit if the next gray value is greater than or equal to the previous one, or a 0 bit if it’s less (each 9-pixel row produces 8 bits of output)</li> <li>Calculate the “column hash”: same as above, but for each column, move top to bottom</li> <li>Concatenate the two 64-bit values together to get the final 128-bit hash</li> </ul></li> <li>Fast: Python is not very fast at bit twiddling, but all the hard work of converting to grayscale and downsizing is done by a C library: ImageMagick+wand or PIL.</li> <li>Available via github: <a href=""></a></li> </ul> <p><strong>#2 Michael:</strong> <a href=""><strong>Google Open Source/Python</strong></a></p> <ul> <li>subprocess32: A reliable subprocess module for Python 2</li> <li>Grumpy: A Python to Go transcompiler and runtime</li> <li>Python Fire: Automatically turns any Python object or module into a command line interface (CLI)</li> <li>Python Client for Google Maps Services: Python client library for Google Maps API Web services</li> <li>Hyou: Pythonic Interface to manipulate Google Spreadsheet</li> <li>oauth2l: A simple CLI tool to get an OAuth token</li> <li>mock_maps_apis: Small AppEngine application that can mock some of the Google Maps APIs</li> <li>TensorFlow: TensorFlow is a fast, flexible, and scalable open source machine learning library </li> </ul> <p><strong>#3 Brian:</strong> <a href=""><strong>How to Handle Missing Data with Python</strong></a></p> <ul> <li>Jason Brownlee</li> <li>Real-world data often has missing values.</li> <li>Data can have missing values for a number of reasons such as observations that were not recorded and data corruption.</li> <li>Handling missing data is important as many machine learning algorithms do not support data with missing values.</li> </ul> <p><strong>#4 Michael:</strong> <a href=""><strong>hug REST framework</strong></a></p> <ul> <li>Drastically simplify API development over multiple interfaces</li> <li>With hug, design and develop your API once, then expose it however your clients need to consume it (locally, over HTTP, or through the command line)</li> <li>hug is the fastest and most modern way to create APIs on <strong>Python3</strong></li> <li>hug has been built from the ground up with performance in mind. <ul> <li>It is built to consume resources only when necessary</li> <li>compiled with Cython to achieve amazing performance</li> </ul></li> <li>Built in version management</li> <li>Automatic documentation</li> <li>Annotation powered validation</li> <li>Write once. Use everywhere (CLI, Python package, Web API)</li> </ul> <p><strong>#5 Brian</strong> <strong>CLI with Click</strong></p> <ul> <li>I needed a cli interface that apparently fell into the “complex” category, because of subcommands, go figure. argparse ticked me off, so I tried click. It’s a joy to work with.</li> <li><a href=""></a></li> <li><a href=""></a> - Just enough click tutorial to do exactly what I wanted. From 2015, realpython blog. (But argparse tutorial is a bit broken. in 3.5/3.6 😞 )</li> </ul> <p><strong>#6 Michael:</strong> <a href=""><strong>Python's Instance, Class, and Static Methods Demystified</strong></a></p> <ul> <li>From <a href=""></a>, guest post from Dan Bader</li> <li>demystify what’s behind class methods, static methods, and regular instance methods</li> <li>Python 3 by default</li> </ul> <pre><code> class MyClass: def method(self): return 'instance method called', self @classmethod def classmethod(cls): return 'class method called', cls @staticmethod def staticmethod(): return 'static method called' </code></pre> <ul> <li>Instances is clear but static and class are not so much <ul> <li>static and class methods are also available on instances</li> <li>choice between class vs static method (do you want inheritance?)</li> <li>instance methods can also access the class itself through the self.__class__ attribute</li> </ul></li> </ul> <p><strong>Follow ups</strong></p> <p>David Bieber from Google and Python Fire sent us this note: The program noted that Fire has one "heavy" dependency, IPython. Just wanted to chime in with this: we have a <a href="">game plan to remove IPython as a required dependency</a>, but we're not there yet. (Contributions are welcome!)</p> <ul> <li>pdir2 now supports color, <a href=""></a>. With the example given, I’ve added a .pdir2config file and changed doc-color to magenta, and I can see the doc lines in both a white and black terminal.</li> </ul> <p><strong>News from us</strong></p> <p><strong>Brian</strong> </p> <ul> <li><a href="">Python Testing with pytest book</a> page has been spotted in the wild by Jakob Jarosz, <a href=""></a>. That was unexpected and cool to see.</li> </ul> <p><strong>Michael</strong></p> <ul> <li>Launched / launching two new courses at Talk Python Training <ul> <li><a href="">Managing Python Dependencies with pip and Virtual Environments</a></li> <li><a href="">Using and Mastering Cookiecutter</a></li> </ul></li> <li>Course bundles now available: <a href=""></a></li> </ul>
Apr 05, 2017
#19 Put your Python dictionaries in a box and apparently Python is really wanted
<p><strong>#1 Brian: Better Exceptions and Python 3</strong></p> <ul> <li>reddit <a href=""></a></li> <li>Qix (Josh), author of better_exceptions, <a href=""></a></li> <li>paradoxxxzero (Josh &amp; Florian), author of <a href=""></a></li> <li>also added a startup hook for better_exceptions, <a href=""></a></li> </ul> <p><strong>#2 Michael:</strong> <strong><a href="">Stack Overflow 2017 Developer Survey Results</a></strong></p> <ul> <li>Python is #5 total usage</li> <li>Python is #1 most wanted</li> <li>Many excellent graphs, be sure to scroll through</li> </ul> <p><strong>#3 Brian: <a href="">Box : Python dictionaries with recursive dot notation access</a></strong></p> <p><strong>#4 Michael: <a href="">Python 3.6.1 is released</a></strong> - Many small improvements and bug fixes. </p> <p><strong>#5 Brian: <a href="">Essential Statistics for Data Science</a></strong></p> <p><strong>#6 Michael: <a href="">Hy: A wonderful dialect of Lisp that's embedded in Python</a></strong></p> <ul> <li>Since Hy transforms its Lisp code into the Python Abstract Syntax Tree, you have the whole beautiful world of Python at your fingertips, in Lisp form!</li> <li>Listener recommended: <a href=""></a></li> <li>You must try: <a href=""></a></li> <li>This is pretty cool because it means Hy is several things: <ul> <li>A Lisp that feels very Pythonic</li> <li>For Lispers, a great way to use Lisp’s crazy powers but in the wide world of Python’s libraries (why yes, you now can write a Django application in Lisp!)</li> <li>For Pythonistas, a great way to start exploring Lisp, from the comfort of Python!</li> <li>For everyone: a pleasant language that has a lot of neat ideas!</li> </ul></li> </ul> <p>Follow up: <a href="">Removal of beer from readme, pdir2 now looks great at default pypi</a></p>
Mar 27, 2017
#18 Python 3 has some amazing types and you can now constructively insult your shell!
<p><strong>#1 Brian: <a href="">pdir2</a></strong></p> <ul> <li>Nice use of animated gif to showcase what it does.</li> <li>It’s a replacement for <code>dir()</code> to use interactively.</li> <li><code>pip install pdir2</code> , but <code>import pdir</code> .</li> <li><code>pdir(something)</code> gives you all that <code>dir()</code> does, but splits things into categories like exceptions, functions, attributes, … </li> <li>each item on one line, and includes the first line of the docstring for the item.</li> <li>Also, uses colors nicely. (Except I need to run it in a shell with non-black background on my mac or I can’t see the docstring output. )</li> <li>Hugely useful if you use <code>dir()</code> interactively. </li> <li>😞 Readme is in markdown, pypi still can’t handle that well. Maybe a listener can do a pull request on it to spiff up the pypi page: <a href=""></a></li> <li>Consider pairing this with <a href=""><strong>ptpython</strong></a> </li> </ul> <p><strong>#2 Michael:</strong> <a href=""><strong>How to recover lost Python source code if it's still resident in-memory</strong></a></p> <ul> <li>Ooops: I screwed up using git ("git checkout --" on the wrong file) and managed to delete the code I had just written, but it was still running in memory.</li> <li>Uses <ul> <li><a href=""></a> Tools for injecting code into running Python processes</li> <li><a href=""></a> A Python cross-version decompiler</li> </ul></li> <li>Main take-away: Really cool to attach <strong>pyrasite</strong> and explore a running Python process for many reasons.</li> </ul> <p><strong>#3 Brian:</strong> <a href=""><strong>New Interesting Data Types in Python 3</strong></a></p> <ul> <li><code>types.MappingProxyType</code> - acts like a dictionary, but it’s read only. Convenient for exposing a mutable data structure through an API and making it less convenient for the client code to modify things they aren’t supposed to.</li> <li><code>types.SimpleNamespace</code>- kind of a general purpose class with attributes defined by the constructor parameters you pass in. May be useful in places where you would use <code>collections.namedtuple</code> . </li> <li><code>typ``ing.NamedTuple</code> - You define a class derived from <code>NamedTuple</code>, and define attributes with types and optionally a default value. Constructor automatically assigns the attributes in order.</li> <li>These types help you to create quick small classes/types allow concise and readable code.</li> </ul> <p><strong>#4 Michael:</strong> <a href=""><strong>howdoi</strong></a></p> <ul> <li>Instant coding and shell answers via the command line </li> <li>Examples <ul> <li>howdoi print stack trace python</li> <li>howdoi connect sqlalchemy</li> <li>howdoi python save dict</li> <li>howdoi debug python</li> <li>howdoi install ohmyzsh</li> <li>howdoi change path macos</li> </ul></li> <li>Notable related <ul> <li><a href=""></a></li> </ul></li> </ul> <p><strong>#5 Brian: A python project from a listener of the show converts to asyncio and speeds up by 150x</strong></p> <ul> <li>Project is a Python interface to a commercial cloud based CRM called Emarsys. But the specifics are kinda beside the point.</li> <li>Comment from episode 17: <a href=""></a> From Diego Mora Cespedes</li> </ul> <p>_Another awesome episode, thanks Michael and Brian!_</p> <p>_About asyncio being awesome, I had my own experience. I had to send information about hundreds of thousands of users to a CRM through their public API daily. Without asyncio, it would have taken 50 hours daily, which we all know is just not possible! After developing a sync (using requests) and async (using aiohttp) client for their API, I managed to send the information about the users to the CRM asynchronously, and it takes... ... ... wait for it... ... ... 20 minutes!_</p> <p>_So that's 150 times faster than without asyncio!_</p> <p>_Anyway, if you wanna take a look at the client I open sourced, here is the link: <a href=""></a>_</p> <p>_Oh yeah, fun fact: the first time I implemented the async functionality, I made too many API calls at the same time, my mac crashed and I think I DDoSed the CRM. Now I use semaphores, which allow you to limit the number of tasks you launch at the same time. So much Python awesomeness!_</p> <ul> <li>However, I can’t find where in this code he uses semaphores, so here’s an example that does use semaphores to limit how many connections to make at a time. <ul> <li><a href="">Making 1 million requests with python-aiohttp</a></li> </ul></li> </ul> <p><strong>#6 Michael:</strong> <a href=""><strong>cookiecutter-pyramid-talk-python-starter</strong></a></p> <p>An opinionated Cookiecutter template for creating Pyramid web applications starting way further down the development chain. This cookiecutter template will create a new Pyramid web application with email, sqlalchemy, rollbar, and way more integrated.</p> <ul> <li><strong>Factored and organized</strong>: This code was generalized out of a large, professional web application</li> <li><strong>Master layout template</strong>: Comes pre-configured with a master layout template. Navigation, CSS, JS, etc is factored into a single template file and is reused across all views</li> <li><strong>Chameleon language</strong>: This template uses the chameleon template language (the cleanest template language for Python - we did say opinionated right?)</li> <li><strong>Pyramid Handlers</strong>: Code is factored into handler / controller classes. You have the full power of object-oriented programming immediately available</li> <li><strong>Secure user management</strong>: The app comes with full user management. Users can register, log in and out, and reset passwords. We use the passlib package for secure user storage using best practices SQLAlchemy data access: Comes with SQLAlchemy ORM preconfigured using sqlite</li> <li><strong>Bootstrap and modern design</strong>: As you can see from the screenshot below, the app comes with bootstrap and fontawesome. It uses a free, open-source theme and images used under CC-Attribution.</li> <li><strong>Logging with LogBook</strong>: A logging system for Python that replaces the standard library’s logging module. It was designed with both complex and simple applications in mind and the idea to make logging fun</li> <li><strong>Runtime error monitoring with Rollbar</strong>: Rollbar adds runtime notifications and detailed error tracking and it comes pre-configured in this template</li> <li><strong>Mailing list integration</strong>: Comes with Mailchimp integration. Just enter your API key and list ID to start collecting and managing users for your mailing list</li> <li><strong>Outbound email with templates</strong>: The app has a set of static HTML files with placeholders that are loaded by the outbound email system and populated with user data</li> <li><strong>Bower static resource management</strong>: Most templates are based on out-of-date files (css templates, js, etc.). This template is uses bower for it's static files. This means a single CLI command will get you the latest everything.</li> <li><strong>Fast pages that are never stale</strong>: Every static resource is referenced with our own cache busting system. This means you can use extremely aggressive caching for performance on static files yet they immediately invalidate upon changes</li> <li><strong>Comes with an entire online course</strong>: This template is built from the final project in Python for Entrepreneurs, a 20 hour course on building professional web apps in Python and Pyramid from Talk Python Training</li> </ul>
Mar 21, 2017
#17 Google's Python is on fire and Simon says you have CPU load Pythonically
<p><strong>#1 Brian: <a href="">python-fire</a></strong></p> <ul> <li>Suggested by several listeners</li> <li>Under the Google repo set on github but not a Google product.</li> <li>“Python Fire is a library for automatically generating command line interfaces (CLIs) from absolutely any Python object.”</li> </ul> <p><em>Some Benefits as listed on the project page</em></p> <ul> <li>a simple way to create a CLI in Python. </li> <li>a tool for exploring and debugging Python code. </li> <li>exploring existing code by turning other people's code into a CLI.</li> <li>makes transitioning between Bash and Python easier. </li> </ul> <p>My take: </p> <ul> <li>Enough documentation right in the github repo for me to try it out. </li> <li>Concise but thorough documentation, as well.</li> <li>I wouldn’t ship a CLI with this, as it’s too heavy. <ul> <li>depends on ipython and six</li> </ul></li> <li>It would be useful to very quickly throw together a CLI to try out some Python code from bash.</li> <li>For internal development and debugging tools.</li> <li>I think this week I’m going to try to build a few CLI tools for directly sending and receiving commands to some test instruments. </li> </ul> <p><strong>#2 Michael:</strong> <a href=""><strong>Simon: Simple macOS menubar system monitor</strong></a><strong>, written in Python 3.6 + pyobjc</strong></p> <ul> <li>Shows how simple a menubar app can be</li> <li>Nice example of a platform-native Python app (we need more of these)</li> <li>Could use it as a starter app for your ideas on macOS</li> </ul> <p><strong>#3 Brian:</strong> <strong>Free Food</strong> A couple of amusing Reddit posts about free food.</p> <ul> <li><a href="">Finding Free Food with Python</a></li> <li><a href="">Notification when friends order pizza</a> </li> </ul> <p><strong>#4 Michael:</strong> <a href=""><strong>HTTPie</strong></a></p> <ul> <li>Pronounced aitch-tee-tee-pie</li> <li>A command line HTTP client with an intuitive UI, JSON support, syntax highlighting, wget-like downloads, plugins, and more.</li> <li>A picture is worth many words, have a look:</li> <li>Excellent support for JSON, XML, HTTP response headers, etc</li> <li>I spoke before about <a href="">Postman</a> as an API test client, this is the CLI version.</li> </ul> <p><strong>#5 Brian: <a href="">pipdeptree</a></strong></p> <ul> <li>Sometimes when doing <code>pip list</code> I see way more packages than I remember installing. That’s due to dependent installs. </li> <li><code>pipdeptree</code> is a simple command line tool that shows you your installed packages in an ascii tree structure so you can see who depends on.</li> <li>Example. <code>arrow</code> ? I don’t remember installing that. Ah. <code>arrow</code> is a dependency of <code>jinja2-time</code> ,which is a dependency of <code>cookiecutter</code> , that I do remember installing.</li> </ul> <p><strong>#6 Michael:</strong> <a href=""><strong>Not Your Father’s Python: Amazing Powerful Frameworks</strong></a></p> <ul> <li>When we were getting SignifAI off the ground, one of the biggest decisions we had to make right at the beginning was what our stack would be. <ul> <li>We know Python but…</li> <li>But it’s important to note that our product and infrastructure must support hundreds of thousands of events per second.</li> <li>So we were happy to see that, with the recent widespread adoption of Python 3 and the introduction of tasks and coroutines as first-class citizens in the language, Python has recently stepped up its game.</li> <li>Python 3 has continued evolving into a new wave of libraries that disrupt and change old assumptions about Python performance for web applications.</li> <li>Python’s GIL, is it a true roadblock?</li> <li>UVLoop is the first ultra-fast asynchronous framework, which is a drop-in replacement for Python 3.5’s built-in asyncio event loop. Both Japronto and Sanic which are reviewed in this post are also based on UVLoop.</li> <li>The future of Python is here: Overall, it looks like fast, asynchronous Python might be here to stay. Now that asyncio appears to be a default in Python and the async/await syntax has found favor among developers, the GIL doesn’t seem like such a roadblock anymore and speed doesn’t need to be a sacrifice.</li> </ul></li> <li>Comment: </li> <li>Sasha Cuerda‏’s tweet: <em>@mkennedy @brianokken Just used the aiohttp example you talked about to refactor a CLI scraper...went from &gt; 8' to &lt; 45". Blown away.</em></li> </ul>
Mar 18, 2017
#16 Postmodern Python and Open-source Financial Awards
<p>This is Python Bytes, Python headlines and news delivered directly to your earbuds: episode 16, recorded on March 6th, 2017. </p> <p>This episode was brought to you by <a href=""><strong>Rollbar: they help you take the pain out of errors</strong></a>.</p> <p><strong>#1 Brian</strong>: <a href=""><strong>Postmodern Error Handling in Python 3.6</strong></a></p> <ul> <li>On "Journal Panic"</li> <li>An amusing look at error prevention <ul> <li>Enum class usage</li> <li>NamedTuple class usage</li> <li>type hints</li> <li>mypy</li> </ul></li> <li>Bonus, another recent pretty darn good mypy walkthrough: <a href="">Python Type Annotations</a> </li> </ul> <p><strong>#2 Michael:</strong> <a href=""><strong>Mozilla Awards $585,000 to Nine Open Source Projects in Q2 2016</strong></a></p> <ul> <li>One of the new tracks is “<a href="">Mission Partners</a>”, which supports any open source project which meaningfully advances the Mozilla mission. We had a large number of applications in the initial round, of which we have <a href="">already funded eight</a> (for a total of $385,000) and are still considering several more. Applications for “Mission Partners” <a href="">remain open</a> on an ongoing basis.</li> <li>The second is our “<a href="">Secure Open Source</a>” track, which works on improving the security of open source software by providing manual source code audits for important and widely-used pieces of free software.</li> <li>Our initial track, “<a href="">Foundational Technology</a>”, which supports projects that Mozilla already uses, integrates or deploys in our infrastructure, was launched late last year and remained open during this quarter. <ul> <li>We made one additional award – to PyPy, the Python JIT compiler, for $200,000. </li> <li>Applications for a “Foundational Technology” award <a href="">remain open</a>.</li> </ul></li> </ul> <p><strong>#3 Brian:</strong> <a href=""><strong>Intel Distribution for Python 2017 Update 2 accelerates five key areas for impressive performance gains</strong></a></p> <ul> <li>“Benchmarks for all these accelerations will be published soon. “</li> <li>"..widespread optimizations for NumPy and SciPy FFT." "..performance may improve up to 60x over Update 1 and is now close to native C/Intel MKL."</li> <li>"Arithmetic and transcendental expressions" from NumPy can now use<code>umath</code> primitives which enables support for all available cores.</li> <li>Memory management optimizations</li> <li>Faster Machine Learning with Scikit-learn</li> <li>Looks like there is a free standalone version and a paid version.</li> <li>Supports Python 2.7 and 3.5. (When’s 3.6 coming, folks?)</li> <li>Supports Windows, Linux, OSX</li> </ul> <p><strong>#4 Michael:</strong> <a href=""><strong>Async HTTP benchmarks on PyPy3</strong></a></p> <ul> <li>Thanks for the heads up <a href="">Guy Fighel, @guyfig</a> <ul> <li>Since <a href="">Mozilla announced funding</a>, we've been working quite hard on delivering you a working Python 3.5.</li> <li>We are almost ready to release an alpha version of PyPy 3.5</li> <li>Pyston faded and PyPy is surging. </li> <li>To show that the heart of Python 3 (asyncio) is already working we have prepared some benchmarks.</li> <li>HTTP workload on several asynchronous IO libraries, namely the relatively new <em>asyncio and</em> <em>curio libraries</em> and the battle-tested <em>tornado</em>, <em>gevent and Twisted libraries</em></li> <li>The purpose of the presented benchmarks is showing that the upcoming PyPy release is already working with unmodified code that runs on CPython 3.5</li> <li>Summary: 5-10x improvement across the board</li> </ul></li> </ul> <p><strong>#5 Brian</strong>: <a href=""><strong>A tale of two exceptions, part 1</strong></a>, <a href=""><strong>and part2</strong></a></p> <ul> <li>Ned Batchelder tried to get the <code></code> test suite to run on Jython.</li> <li>Jython doesn't have whatever coverage needs for the reporting part of coverage, so Ned went through some attempts to get the exceptions working ok and try to skip tests that required reporting.</li> <li>Headaches start and raise lots of design questions with a normal set of pragmatic decisions that have to be made. And following Ned's thought process is a cool look at problem solving.</li> <li>The first post ends with a solution that includes production code raising test exceptions.</li> <li>The second post replaces that with a solution that uses a metaclass to apply a decorator to each test function at object instantiation. The decorator looks for a custom exception and raises SkipTest. </li> </ul> <p><strong>#6 Michael:</strong> <a href=""><strong>Asynchronous HTTP Requests in Python</strong></a></p> <ul> <li>If you’re familiar with the popular Python library <code>requests</code> you can consider <code>aiohttp</code> as the asynchronous version of <code>requests</code>.</li> <li>Usage is very similar to <code>requests</code> but the potential performance benefits are, in some cases, absolutely insane.</li> <li>Requests: It’s a fairly straightforward program and takes around 12 minutes of total time.</li> <li><code>aiohttp</code>'s <code>ClientSession.get()</code> and <code>aiofiles</code>: execution time is about <em>22 seconds</em>, which in my opinion is a ridiculous performance boost (that’s 33x faster).</li> </ul>
Mar 07, 2017
#15 Digging into Python packaging
<p>This is Python Bytes, Python headlines and news delivered directly to your earbuds: episode 15, recorded on February 27, 2017. </p> <p><strong>#1 Brian:</strong> <strong>Packaging</strong> <a href=""><strong>A Simple Guide for Python Packaging</strong></a> <strong><em>*by *</em></strong>Jie Feng, <a href="">@flyfengjie</a></p> <ul> <li>very simple bare bones example and tutorial</li> <li>When you create a ‘.py’ file, that is a module.</li> <li>One or more modules in a folder with add a file is a package (named via the folder)</li> </ul> <p><a href=""><strong>How To Package Your Python Code</strong></a> <em>*</em>*by Scott Torborg, <a href="">@storborg</a></p> <ul> <li>The example from the previous article comes from this.</li> <li>The best mid level explanation I’ve run across so far to describe packaging and distribution.</li> <li>Read the whole thing in a very short time.</li> </ul> <p><strong>#2 Michael:</strong> <a href=""><strong>elasticpypi by</strong></a> <a href=""><strong>Kyle Hornberg</strong></a></p> <ul> <li>A mostly functional simple pypi service running on AWS.</li> <li>Runs over Elastic Search and AWS Lambda</li> <li>Compare to <ul> <li>pypiserver:</li> <li>devpi:</li> <li>pypiserver:</li> </ul></li> </ul> <p><strong>#3 Brian:</strong> <strong>How to get Python new<em>*</strong>s (our process)</em>*</p> <ul> <li><a href=""></a>, of course</li> <li>Twitter: <ul> <li>Brian: <a href=""></a></li> <li>Michael: <a href=""></a></li> <li>Talk Python: <a href=""></a></li> <li>Python Bytes: <a href=""></a></li> </ul></li> <li><a href=""></a>, check out the titles only link, as well as reading this feed with Feedly or some other feed reader</li> <li>Newsletters: <ul> <li>Awesome Python: <a href=""></a></li> <li>Python Weekly: <a href=""></a></li> <li>Pycoders Weekly: <a href=""></a></li> <li>Import Python: <a href=""></a></li> </ul></li> <li>Reddit: <a href=""></a></li> <li>Python Trending on GitHub: <a href=""></a></li> </ul> <p><strong>#4 Michael:</strong> <a href=""><strong>Curio - The coroutine concurrency library</strong></a><strong>, by</strong> <strong>David Beazley,</strong> <a href=""><strong>@dabeaz</strong></a></p> <ul> <li>Curio is a library for performing concurrent I/O and common system programming tasks such as launching subprocesses and farming work out to thread and process pools.</li> <li>It uses Python coroutines and the explicit async/await syntax introduced in Python 3.5.</li> <li>Based on cooperative multitasking and existing programming abstractions such as threads, sockets, files, subprocesses, locks, and queues.</li> <li>Only works on POSIX systems</li> <li>The Big Question: Why? <ul> <li>Python 3.4 and 3.5 have added major new paradigms for async programming.Curio takes full advantage of these features and is not encumbered by issues of backwards compatibility with legacy Python code written 15 years ago. <ul> <li>Curio is a ground-up implementation that takes a different approach to the problem while relying upon known programming techniques involving sockets and files.</li> <li>The implementation of Curio aims to be simple. The API for using Curio aims to be intuitive.</li> </ul></li> </ul></li> </ul> <p><strong>#5 Brian:</strong> <a href=""><strong>Pandas switching to use pytest as testing framework</strong></a></p> <ul> <li>Interesting look at the challenges and discussion of moving a large, highly visible project</li> </ul> <p><a href=""><strong>#6 Michael: Talk Python #100: Python past, present, and future with Guido van Rossum</strong></a></p> <ul> <li>Excellent look back and forward with Guido</li> <li>Conversation <a href="">on reddit</a></li> <li>Conversation <a href="">on hackernews</a></li> <li>We touch on <ul> <li>How he got started</li> <li>Early influences on Python</li> <li>Why Python succeeded</li> <li>How are we doing with: Diversity and moving to Python 3?</li> <li>His favorite features of Python 3</li> <li>Converting legacy code via mypy: <a href=""></a></li> </ul></li> </ul> <p><strong>In other news (as in our news)</strong></p> <ul> <li><p><a href="">Test &amp; Code 27 is out, a great talk with Mahmoud Hashemi</a> on different levels of testing, the role of testing in SW development, TDD, and what he’s been up to lately. Test &amp; Code migrating to <a href=""></a>, to separate writing and podcast into two sites. When it gets cleaned up a little, I’ll forward podcast related links from <a href="">pythontesting</a> to <a href="">testandcode</a>.</p></li> <li><p><a href="">The pytest book</a> got sent out to a few reviewers, and we have a title and some cover art options. It’s getting exciting. The rest of the reviewers will get the book when all chapters have gone through at least one editor iteration.</p></li> </ul>
Feb 28, 2017
#14 Lots of Python style and Python 3000 is 3000 days old
<p><strong>Brian #1:</strong> <a href=""><strong>Tiny Python 3.6 Notebook - Matt Harrison</strong></a></p> <ul> <li>README : <a href=""></a></li> <li>Full Book on GitHub : <a href=""></a></li> <li>“This is not so much an instructional manual, but rather notes, tables, and examples for Python syntax.”</li> <li>Physical book is 134 pages, but looks like about 90 pages of content. </li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>Oh no! This package is Python 2 only - Anthony Shaw</strong></a></p> <ul> <li>You’re head down, working on a new project and one of your dependencies still doesn’t support Python 3 - argh! Here’s a quick guide on how to solve that problem, step by step.</li> <li>Step 0: Check that nobody else has solved this</li> <li>Step 1: Fork it</li> <li>Step 2: Print statements</li> <li>Step 3: Tests <ul> <li>Static Analysis: modernize</li> </ul></li> <li>Step 4: Update</li> <li>Step 5: Install it into your original project</li> <li>Step 6: Raise a pull request</li> <li>Step 7: 3 months later?</li> </ul> <p><strong>Brian #3:</strong> <a href=""><strong>Elements of Python Style</strong></a> <strong>- Andrew Montalenti</strong></p> <ul> <li>More than PEP8, and opinionated. </li> <li>I admire the effort, and I may at some point fork it to remove/fix the few things I disagree with and to be able to use it as a style guide for my team.</li> <li>Great: most of it. <ul> <li>Use parens for continuation</li> <li>Use with for files and locks</li> </ul></li> <li>May need tweaking <ul> <li>Avoid custom exceptions</li> <li>I’d prefer, “Add one custom base exception, and build specific exceptions off of that”. </li> <li>This may be a controversial point, or I may be just confused about conventional practice here</li> <li>As with any style guide, more detail brings more controversy. </li> </ul></li> <li>Michael: I like this guidance: You can also choose to use CamelCase for things that are class-like but not quite classes -- the main benefit of CamelCase is calling attention to something as a "global noun", rather than a local label or a verb. Notice that Python names True, False, and None use CamelCase even though they are not classes.</li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>Python 3 was exactly 3000 days old this past Sunday</strong></a></p> <ul> <li>Reddit post about my tweet (wow) <ul> <li>537 upvotes, 71 comments</li> </ul></li> <li>Test it yourself: background = “Python 3 was released December 3, 2008 ” + \ “Its original working name was Python 3000” release =, month=12, day=3) today =, month=2, day=19) (today - release).days # → 3000</li> <li>Via @cclauss Christian Clauss</li> </ul> <p>Brian #5: <strong>From beginner to pro:</strong> <a href=""><strong>Python books, videos and resources</strong></a> <a href=""></a></p> <ul> <li>Bob Belderbos and Julian Sequeira</li> <li>Reddit thread: <a href=""></a></li> <li>Post: <a href=""></a></li> <li>“a list of useful Python resources to boost up your skills”</li> <li>Some very nice podcasts listed in “other resources”</li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>mongoaudit</strong></a></p> <ul> <li>mongoaudit is a CLI tool for auditing MongoDB servers, detecting poor security settings and performing automated penetration testing.</li> <li>pip install mongoaudit</li> <li>Tests things like: <ul> <li>MongoDB listens on a port different to default one</li> <li>Server only accepts connections from whitelisted hosts / networks</li> <li>MongoDB HTTP status interface is not accessible on port 28017</li> <li>MongoDB is not exposing its version number</li> <li>MongoDB version is newer than 2.4</li> <li>TLS/SSL encryption is enabled</li> <li>Authentication is enabled</li> <li>SCRAM-SHA-1 authentication method is enabled</li> <li>Server-side Javascript is forbidden</li> <li>Roles granted to the user only permit CRUD operations</li> <li>The user has permissions over a single database</li> <li>Various security vulnerabilities </li> </ul></li> <li>From Michael: MongoDB is awesome. But please make sure at least one of the following is true. <ul> <li>You only listen on local loopback ( and run mongo on the web server</li> <li>You run mongodb with authentication enabled (it’s off by default)</li> <li>You run mongodb with SSL enabled (may be off by default too)</li> </ul></li> </ul> <p>What else? Launched my course at <a href=""></a></p>
Feb 22, 2017
#13 Python making the move to GitHub and Dropbox is stepping back from Pyston
<p>This is Python Bytes, Python headlines and news deliver directly to your earbuds: episode 13, recorded on February 13, 2017. In this episode we discuss Python making the move to GitHub and Dropbox stepping back from Pyston. </p> <p>This episode was brought to you by <a href="">Metis: The Data Science Bootcamp company</a>.</p> <p><a href=""><strong>#1 Brian:Pyston no longer sponsored by Dropbox</strong></a></p> <p>Personal follow up post by Kevin Modzelewski <a href=""></a></p> <ul> <li>Pyston (pronounced piston) is a Python JIT implementation started at Dropbox</li> <li>It was based on CPython and supported a bunch of 2.7, but wasn’t complete.</li> <li>Bottom line: It’s open source, and the repo will be left for whoever wants to work on it. But the core developers from Dropbox won’t be working on it, and Dropbox won’t be spending any more time/money on it.</li> </ul> <p><a href=""><strong>#2 Michael: CPython is coming to GitHub</strong></a></p> <ul> <li>Mailing list announcment: <a href=""></a></li> <li>Reddit discussion <a href=""></a></li> </ul> <p>Brett Cannon’s excellent background story: <a href=""></a></p> <ul> <li>Interesting that some people (voiced via reddit) sadness about leaving Hg…</li> <li>2006: Python moves to SVN</li> <li>2011: Python moves to Hg</li> <li>2017: Python moves to GitHub</li> <li>By 2014 it had become obvious to some of us that the Python development process had in fact become a burden. The rate at which patches were being submitted was much greater than the rate at which they were being reviewed. This was leading to external contributors getting frustrated because they would put in the effort to write a patch but would occasionally end up with waiting years for a review from a core developer.</li> <li>I wanted was the ability to review an external contribution -- from submission to commit -- all on a tablet while at a beach with WiFi (which I actually have in Vancouver so this wasn't entirely a silly request). My thinking was that if we got the process to be that simple, core developers could do a review at lunch time while at work, or when they had some down time at home without having to be on some special machine that had their SSH keys installed on it.</li> </ul> <p><a href=""><strong>#3 Brian: Using functional programming in Python like a boss: Generators, Iterators and Decorators</strong></a></p> <ul> <li>I’m liking the trend of more Jupiter notebook based articles. This one is a pretty gentle introduction into functions, generators, iterators, decorators, containers, and how they all work together to make your code more expressive.</li> </ul> <p><a href=""><strong>#4 Michael: It's metaclasses all the way down</strong></a></p> <ul> <li>What is metaprogramming? Metaprogramming is a technique of writing computer programs that can treat themselves as data, so you can introspect, generate, and/or modify them while running</li> <li>Metaclasses are deeper magic than 99% of users should ever worry about. If you wonder whether you need them, you don’t (Tim Peters)</li> <li>However: The potential uses for metaclasses are boundless. Some ideas that have been explored include logging, interface checking, automatic delegation, automatic property creation, proxies, frameworks, and automatic resource locking/synchronization. </li> <li>Nice examples: <ul> <li>Django ORM models</li> <li>Abstract vehicle class</li> </ul></li> </ul> <p><a href=""><strong>#5 Brian: Lambda Functions in Python: What Are They Good For?</strong></a></p> <ul> <li>Lambdas are part of the language, and if used sparingly and in the right context, can make your code more readable. It’s super important for intermediate and experienced Python folks to understand lambdas and not be afraid of them.</li> <li>This article is a good tutorial on them.</li> <li>When you finally grok them, you might be tempted to use them all over the place. Dan presents a couple of places where using lambdas is a bad choice: <ul> <li>creating class methods. Definitions are definitely preferred over assigning lambdas to attributes. </li> <li>generating a list with <code>list(fliter(&lt;lambda expression&gt;, &lt;iterable&gt;))</code> . I agree with Dan that comprehensions are way easier to read.</li> </ul></li> </ul> <p><a href=""><strong>#6 Michael: Multi-threaded SQLite without the OperationalErrors</strong></a></p> <ul> <li>Unless you are very diligent about keeping your write transactions as short as possible, you can easily wind up with one thread accidentally holding a write transaction open for an unnecessarily long time. </li> <li>I've had success using a very simple approach: I eliminate the possibility of lock contention by dedicating one thread to the task of issuing all writes for the application.</li> <li>The entire source listing can be found here, if you're curious: <a href=""></a></li> </ul> <p><a href=""><strong>Followup:</strong></a> <a href=""><strong>J</strong></a><a href=""><strong>apronto not production ready</strong></a>: “Japronto author here: As stated in the README you should not build anything serious with Japronto now because it's gonna probably eat your laundry. Remember it's hand coded in C and this needs a lot of testing. On the top of that I plan to do several iterations of API changes in a largely incompatible ways. I hope though it's gonna make a serious player in the field one day.</p> <p>If you wanna a decent async framework then go with aiohttp or Sanic. If you are gonna do typical REST app frontend to a database go with Flask or Pyramid. If you need a scaffolded admin go with Django.”</p>
Feb 14, 2017
#12 Expanding your Python mental model and serving millions of requests per second with Python
<p>This is Python Bytes, Python headlines and news deliver directly to your earbuds: episode 12, recorded on February 6th, 2017. In this episode we discuss expanding your Python mental model and serving millions of requests per second with Python. </p> <p>This episode was brought to you by <a href="">Rollbar: they help you take the pain out of errors</a>.</p> <p><strong>BO 1: A couple of mental model articles</strong></p> <ul> <li><strong>Python Functions aren’t what you think.</strong> <ul> <li></li> <li>Functions are objects. The name of a function is just a variable name referring to the object.</li> </ul></li> <li><strong>The Tao of Python</strong> <ul> <li></li> <li>Jupiter notebook discussion of the relationship between objects, classes, and metaclasses.</li> </ul></li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>Why Learn Python? Here Are 8 Data-Driven Reasons</strong></a> <strong>by Dan Bader</strong></p> <ul> <li>Is Python worth learning? We’ve interviewed experts and surveyed the job market to identify the key reasons why you should learn Python today.</li> <li>Python is also one of the hottest skills to have according to <a href="">research by Dice</a></li> <li>The 2nd most popular programming language in the world based on the <a href="">PYPL Popularity of Programming Language Index</a>. <ul> <li><ol> <li>You Can Use Python for Pretty Much Anything</li> </ol></li> <li><ol> <li>Python Is Widely Used in Data Science</li> </ol></li> <li><ol> <li>Python Pays Well: Indeed’s salary calculator gives an even larger figure—a whopping $116,000 per year. (second only to ruby according to gooroo but has 3 times the job openings)</li> </ol></li> <li><ol> <li>Demand for Python Developers Is High (And Growing)</li> </ol></li> <li><ol> <li>Python Saves Time</li> </ol></li> <li><ol> <li>Python Is Beginner Friendly</li> </ol></li> <li><ol> <li>All the Big Names Use Python</li> </ol></li> <li>Python could be your way into major tech companies: YouTube, IBM, Yahoo, Dropbox, Quora, Mozilla, Instagram, and many others </li> <li><ol> <li>Python Has an Amazing Ecosystem</li> </ol></li> <li>Segments from: Michael Kennedy, Ankur Gupta (Curator at ImportPython), and Sebastian Vetter (Python Engineer at Eventbase)</li> </ul></li> </ul> <p><strong>B<em>*</strong>rian #<strong></em>*3:</strong> <a href=""><strong>Testing Python Applications with Pytest - Kevin Ndung’u</strong></a> How could I resist? Do a decent job explaining something about pytest, and I’m happy. What I liked:</p> <ul> <li>good really fast intro to pytest extreme basics</li> <li>Refactoring test code with fixtures. Pulling out common starting state into a fixture.</li> <li>Paramterizing tests to send multiple data sets into the same test function</li> <li>Using both fixtures and paratmerization in the same test function</li> </ul> <p>There’s so much power and functionality in pytest. But just starting to use it as a better test framework for Python is a good thing. This article is a good kick in the pants for someone new to pytest. Easy intro, plus a couple of cool goodies.</p> <p><strong>Michael #4:</strong> <a href=""><strong>A million requests per second with Python</strong></a></p> <ul> <li>Screaming-fast Python 3.5+ web micro-framework integrated with pipelining HTTP server based on uvloop and picohttpparser.</li> <li>We spoke about sanic recently. Apparently this is much faster.</li> <li> 2.6k stars</li> <li>Is it possible to hit a million requests per second with Python? Probably not until recently.</li> <li>Covers perf improvements found in 3.6 and coming in 3.7.</li> <li>It lets you do both synchronous and asynchronous programming thanks to asyncio. And it’s shamelessly fast. Even faster than NodeJS and Go.</li> <li>Japronto tries hard to delay creation of Python counterparts of its internal structures until asked explicitly. For example, a headers dictionary won’t be created until it’s requested in a view.</li> </ul> <p><strong>Brian#5:</strong> <a href=""><strong>RethinkDB is alive and well</strong></a></p> <ul> <li>RethinkDB joins The Linux Foundation <ul> <li></li> </ul></li> <li>The liberation of RethinkDB <ul> <li></li> </ul></li> <li>Company behind RethinkDB shut down Sept 2016</li> <li>Announced today that Cloud Native Computing Foundation purchased the rights to source code and contributed it to the Linux Foundation under Apache license ASLv2.</li> <li>Website, GitHub org, social media accounts will continue to operate.</li> <li>They can take donations, and stripe has ponied up 25k in donation matching.</li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>Python Top 10 Articles for the Past Year (v.2017)</strong></a></p> <ol> <li>The Hitchhiker’s Guide to Python: Best practices guidebook written for Humans.</li> <li>Scipy Lecture Notes — Learn numerics, science, and data with Python.</li> <li>30 Essential Python Tips and Tricks for Programmers.</li> <li>Computational and Inferential Thinking for Data Science. Courtesy of UC Berkeley</li> <li>Welcome to Python cheatsheet.</li> <li>Data Mining in Python: A Guide.</li> <li>Python FAQ: Why should I use Python 3? (by eevee)</li> <li>An Introduction to Stock Market Data Analysis with Python</li> <li>NumPy Tutorial: Data analysis with Python.</li> <li>Build Your First Python and Django Application</li> </ol> <p><strong>Michael: Follow up on RHL:</strong> <a href=""><strong>Red Hat Software Collections</strong></a> Thank you <a href="">Chip Warden</a></p> <ul> <li>The latest, stable updates of development technologies for Red Hat Enterprise Linux</li> <li>Add latest for: <ul> <li>Node </li> <li>Perl </li> <li>PHP </li> <li>Python [YES! Python 3]</li> <li>Ruby</li> </ul></li> </ul>
Feb 07, 2017
#11 Django 2.0 is dropping Python 2 entirely, pipenv for profile functionality, and Pythonic home automation
<p>This is Python Bytes, Python headlines and news deliver directly to your earbuds: episode 11, recorded on <strong>January 30th, 2017</strong>. </p> <p><strong>#1 (Brian) <a href="">pipenv</a> - Pipfile, pip, and virtualenv</strong></p> <ul> <li><a href="">announcement from Kenneth Reitz</a> </li> <li><a href="">reddit thread</a> </li> <li>Features <ul> <li>Automatically finds your project home, recursively, by looking for a Pipfile.</li> <li>Automatically generates a Pipfile, if one doesn't exist.</li> <li>Automatically generates a Pipfile.lock, if one doesn't exist.</li> <li>Automatically creates a virtualenv in a standard location (project/.venv).</li> <li>Automatically adds packages to a Pipfile when they are installed.</li> <li>Automatically removes packages from a Pipfile when they are un-installed.</li> <li>Also automatically updates pip.</li> </ul></li> </ul> <p><strong>#2 (Michael):</strong> <a href=""><strong>Django 2.0 is dropping support for legacy Python</strong></a></p> <ul> <li>Django changing docs to default to Python 3</li> <li>The next release, Django 1.11, will be a long-term support release, and the one after that, Django 2.0, will no longer support Python 2.</li> </ul> <p><strong>#3 (Brian)</strong> <a href=""><strong>attrs</strong></a></p> <ul> <li>Hynek Schlawack</li> <li>pypi: <a href=""></a></li> <li>readthedocs: <a href=""></a></li> <li>I know this has been around for a while. But I’ve just stumbled across it while reading a <a href="">blog post about requests</a>, which was good, but we’ve covered requests a lot lately, so I’m gonna skip that article today.</li> <li>pip install attrs, with an s, even though you import without the s</li> <li>Does all of the grunt work of writing dunder functions for you so you can write classes with a small amount of code that behave like classes and objects should. Especially if you come from a C++ background, this makes writing classes more intuitive.</li> </ul> <p><strong>#4 (Michael):</strong> <a href=""><strong>Go faster Python</strong></a></p> <ul> <li>This blog post gives an introduction to some techniques for benchmarking, profiling and optimising Python code.</li> <li>If you have a Python program that’s running slowly, what are your options? <ul> <li>Benchmarking and profiling</li> <li>Our intuition is often wrong</li> <li>Benchmarking: %time, %timeit, timeit</li> <li>Function profiling: %prun, cProfile</li> <li>Line profiling: %lprun, line_profiler (requires <a href="">line_profiler</a>)</li> <li>Cython</li> </ul></li> </ul> <p><strong>#5 (Brian):</strong> <strong>Getting Python 3 into distributions</strong></p> <ul> <li>Not an article but a couple of pleas.</li> <li>Many OS distributions, including Red Hat, ship with Python 2.7.</li> <li>Many developers don’t have the authority to install Python 3.x for projects.</li> <li>Two pleas: <ul> <li>distributions: ship with both if you have to, but let 3.6 be an option for people.</li> <li>companies: install Python 3.6 and let some projects use that</li> </ul></li> <li>We can’t just encourage users to switch to Python 3 if it’s not their choice.</li> </ul> <p><strong>#6 (Michael)</strong> <a href=""><strong>Home Assistant</strong></a></p> <ul> <li>Home Assistant is an open-source home automation platform running on Python 3. <ul> <li>Track and control all devices at home and automate control</li> <li>Installation in less than a minute.</li> <li>Observe: Track the state of all the devices in your home, so you don't have to.</li> <li>Control: All your devices from a single, mobile-friendly, interface.</li> <li>Automate: Setup advanced rules to control devices and bring your home alive.</li> <li>have the lights turn on when the sun sets and you are home?</li> <li>have the lights turn on when anyone comes home and it is dark?</li> <li>dim the lights when you start watching a movie on your Chromecast?</li> <li>receive a message when the lights turn on while you are not at home?</li> </ul></li> <li>Demo: <a href=""></a></li> <li><a href="">aiohttp</a>: Asynchronous HTTP Client/Server</li> </ul>
Jan 31, 2017
#10 Dismissing Python's Garbage Collection, PyPI Name Reservations, and Hackers Exfiltrate US Government Data to Save Itself
<p>This is Python Bytes, Python headlines and news deliver directly to your earbuds: episode 10, recorded on <strong>Monday, January 23rd, 2016</strong>. </p> <p><strong>#1 (Brian): <a href="">Advanced Time Series Plots in Python</a> from</strong></p> <ul> <li>matplotlib 2.0 official release on Jan 17 <ul> <li><a href="">mentioned it was coming in episode 7</a></li> <li><a href=""></a></li> </ul></li> <li>A tutorial for using pandas, numpy, matplotlib, and seaborn for collecting, analyzing, and plotting time series plots. <ul> <li>start with an empty xy chart</li> <li>grab some yahoo finance data, do some manipulation, and plot it.</li> <li>add shaded areas, in the example, recession times</li> <li>chart titles</li> <li>axis labels</li> <li>legend styling</li> <li>horizontal line, at the 0 value, in this example</li> <li>formatting x and y tick labels</li> <li>controlling the font</li> <li>turning on data points by specifying a marker shape</li> <li>add some chart annotations with arrows pointing to specific data points</li> <li>add a logo and watermark over the top of everything</li> </ul></li> <li><a href="">seaborn</a> is a statistical data visualization tool for use with matplotlib</li> <li>See the seaborn site has tutorials for: <ul> <li>Style functions, Color palettes, Distribution plots, Regression plots, Categorical plots, and Axis grid objects. </li> </ul></li> </ul> <p><strong>#2 (Michael):</strong> <a href=""><strong>Dismissing Python Garbage Collection at Instagram</strong></a></p> <ul> <li>By dismissing the Python garbage collection (GC) mechanism, which reclaims memory by collecting and freeing unused data, Instagram can run 10% more efficiently.</li> <li>How We Run Our Web Server: Instagram’s web server runs on Django in a multi-process mode with a master process that forks itself to create dozens of worker processes that take incoming user requests. For the application server, we use uWSGI with pre-fork mode to leverage memory sharing between master and worker processes. </li> <li>Attempt 1: Disable reference count on code objects (no effect)</li> <li>Attempt 2: Let’s try disabling GC <ul> <li>With that, we successfully raised the shared memory of each worker process from 140MB to 225MB, and the total memory usage on the host dropped by 8GB per machine. This saved 25% RAM for the whole Django fleet. With such big head room, we're capable of running a lot more processes or running with a much higher RSS memory threshold. In effect, this improves the throughput of Django tier by more than 10%.</li> </ul></li> <li>Attempt 3: Completely shutdown GC takes churns (bad)</li> <li>Attempt 4: Final step for shutting down GC: No cleanup (success)</li> </ul> <p><strong>#3 (Brian): <a href="">My experience with type hints and mypy</a></strong></p> <p>Brett Cannon on his blog <a href=""></a></p> <ul> <li>Here's an example including the pull request of changes, of a project adding static types and static analysis of those types.</li> <li>Trying to use type hints to increase code quality and readability for a project.</li> <li>The project is CLA enforcement bot for the PSF, to make sure that all pull requests contributions to a python project is done by someone who has signed a PSF Contributor Licence Agreement.</li> <li>Process <ul> <li>Run mypy against your code with no types, then slowly add types, one object at a time.</li> <li><a href="">mypy</a> is a static analysis tool to make sure your typehints are correct. </li> </ul></li> <li>Issues: <ul> <li>supports Python 3.5, but not new types in Python 3.6, specifically typying. Collection. Workaround was to use typing.AbstractSet</li> <li>f-strings not supported yet, but coming</li> <li>Skipped type hinting for test suite. </li> <li>It wouldn't occur to me to even try to add static types to tests.</li> <li>Most linters would have caught the problems as well as mypy.</li> </ul></li> <li>Benefit <ul> <li>The act of adding static types increased the readability of the code. Did not detract from readability.</li> <li>“What mypy really got me was better documentation. While I was adding the type hints there were a couple of times where I had to examine the code to realize what the appropriate type was. Now that I have the functions and methods all hinted I don't have to guess anymore. That should make long-term maintenance a bit easier”</li> </ul></li> <li>Conclusion "…would I bother typing new Python 3 code? … yes once mypy supports f-strings. “When I design an API I already have to think about what type of objects would be acceptable, so quickly writing down my assumptions doesn't hurt anything, it's relatively quick, and it benefits anyone having to work with my code. But I also wouldn't contort my code to fit within the confines of type hints (i.e. if type hints forces me to write cleaner code then that's great, but if something is so dynamic that it can't have type hints then that's fine and I'll happily use typing.Any as an escape hatch). “ “In the end I view type hints as enhanced documentation that has tooling to help verify that the documentation about types is accurate. And for that use-case I see type hints worth doing and not at all a burden.”</li> </ul> <p><strong>#4 (Michael):</strong> <a href=""><strong>Understanding the underscore( _ ) of Python</strong></a></p> <ul> <li>The <em>underscore</em> (_) is special in Python.</li> <li>If you are python programmer, for _ in range(10) , __init__(self) like syntax may be familiar.</li> <li>There are 5 cases for using the underscore in Python. <ul> <li>For storing the value of last expression in interpreter.</li> <li>For ignoring the specific values. (so-called “I don’t care”)</li> <li>To give special meanings and functions to name of vartiables or functions.</li> <li>_protected-ish, __<em>private-ish, conflicting: in_</em>, __magic__</li> <li>To use as ‘Internationalization(i18n)’ or ‘Localization(l10n)’ functions.</li> <li>To separate the digits of number literal value.</li> </ul></li> </ul> <p><strong>#5: (Brian): <a href="">PEP 541 -- Package Index Name Retention</a></strong> Draft status currently; was just submitted on the 12th by Donald Stufft. “ Abstract: This PEP proposes an extension to the Terms of Use of the Package Index, clarifying expectations of package owners regarding ownership of a package name on the Package Index, specifically with regards to conflict resolution.</p> <p>Existing package repositories such as CPAN , NPM , and GitHub will be investigated as prior art in this field.</p> <p>Rationale Given that package names on the Index are sharing a single flat namespace, a unique name is a finite resource. The growing age of the Package Index causes a constant rise of situations of conflict between the current use of the name and a different suggested use of the same name.</p> <p>This document aims to provide general guidelines for solving the most typical cases of such conflicts.”</p> <p><strong>#6 (Michael):</strong> <a href=""><strong>Hackers downloaded US government climate data and stored it on European servers as Trump was being inaugurated</strong></a></p> <ul> <li>As Donald Trump was sworn into office as the new president of the US on Jan. 20, a group of around 60 programmers and scientists were gathered in the Department of Information Studies building at the University of California-Los Angeles, harvesting government data.</li> <li>A spreadsheet detailed their targets: <ul> <li>Webpages dedicated to the Department of Energy’s solar power initiative</li> <li>Energy Information Administration data sets that compared fossil fuels to renewable energy sources</li> <li>fuel cell research from the National Renewable Energy Laboratory, to name a few out of hundreds.</li> </ul></li> <li>Volunteer “data rescue” events in Toronto, Philadelphia, Chicago, Indianapolis, and Michigan over the last few weeks have managed to scrape hundreds of thousands of pages off of,,, and, uploading them to the Internet Archive. Another is planned for early February at New York University.</li> <li>Suddenly, at exactly noon on Friday as Trump was sworn in, and just as the UCLA event kicked off, some of their fears began to come true: The climate change-related pages on disappeared.</li> <li>There will, thanks to Michael Riedyk, CEO of the Canadian data-archiving company Page Freezer, also be a copy stored outside the US.</li> </ul>
Jan 23, 2017
#9 Walking with async coroutines, diving deep into requests, and a universe of options (for AIs)
<p>This is Python Bytes, Python headlines and news deliver directly to your earbuds: episode 9, recorded on <strong>Tuesday, January 17th</strong>. In this episode we discuss <strong>walking with async coroutines, diving deep into requests, and a universe of options (for AIs)</strong>. </p> <p><strong>#1 (Brian): <a href="">Python Asynchronous I/O Walkthrough</a></strong></p> <ul> <li>In July, there was an open source book published called <a href="">500 Lines or Less</a>.</li> <li>One of the chapters was called <a href="">A Web Crawler With asyncio Coroutines</a>, written by A. Jesse Jiryu Davis and Guido van Rossum. It explains async networking, showing how non-blocking sockets work and how Python 3’s coroutines improve asynchronous network programs.</li> <li>As mentioned recently on A. Jesse Jiryu Davis’s blog <a href=""></a>, the chapter may be difficult for a novice or intermediate developer to follow.</li> <li>This month, Philip Guo, an Assistant Professor at UC San Diego, has released an 8 part video series, 90 minutes total, so on the order of 10 minutes each, that walks through the article, including some coding examples that he walks through.</li> <li>Since async is something that takes a while to get your head around, I appreciate the multi-sensory education experience. Listening to Philip talk about it, watching him walk through the code and talk about the article, and having the article as a reference, is super helpful.</li> <li><a href="">Talk Python #22: CPython Internals and Learning Python with</a></li> <li><a href="">Talk Python #69: Write an Excellent Programming Blog</a></li> </ul> <p><strong>#2 (Michael):</strong> <a href=""><strong>4 likely future twists for Python</strong></a> by <em>Serdar Yegulalp</em></p> <ol> <li><p><strong>Python 2.x may live on</strong></p> <ul> <li>Python 2.x might also get a continued lease on life if independent developers decide to keep the branch going on their own.</li> <li>At least one such effort exists -- Naftali Harris's "<a href="">Python 2.8</a>" project, which backports improvements and bug fixes from Python 3 into the Python 2.x branch.</li> <li>it makes sense to make the 3.x leap, but it's likely we'll see a lot of keep-the-2.x-flame-alive efforts</li> </ul></li> <li><p><strong>Requirements.txt may be replaced with something better</strong></p> <ul> <li><a href="">Pipfile</a> has been proposed as a possible replacement by the folks at the Python Packaging Authority, which is "the working group that maintains many of the relevant projects in Python packaging."</li> <li>Pipfile will be superior to requirements.txt file in a number of ways: 1. TOML syntax for declaring all types of Python dependencies. 2. One Pipfile (as opposed to multiple requirements.txt files). 3. Existing requirements files tend to proliferate into multiple files - e.g. dev-requirements.txt, test-requirements.txt, etc. - but a Pipfile will allow seamlessly specifying groups of dependencies in one place. 4. This will be surfaced as only two built-in groups (default &amp; development). (see note below) 5. Fully specified (and deterministic) environments in the form of Pipfile.freeze. A deployed application can then be completely redeployed with the same exact versions of all recursive dependencies, by referencing the Pipfile.freeze file.</li> <li><p>Example pipfile: </p> <p>[[source]] url = '' verify_ssl = true</p> <p>[requires] python_version = '2.7'</p> <p>[packages] requests = { extras = ['socks'] } Django = '>1.10' pinax = { git = 'git://', ref = '1.4', editable = true }</p> <p>[dev-packages] nose = '*' </p></li> </ul></li> <li><p><strong>Python could get more enterprise editions</strong></p> <ul> <li>As the language has gained traction across the board, it's also appearing in versions aimed specifically at solving enterprise-grade problems.</li> <li>Intel, for instance, elected to <a href="">repackage</a> the Anaconda science-and-math distribution of Python after outfitting it with extensions that give it a speed boost, albeit only on Intel processors. </li> <li>Anaconda is itself produced by Continuum Analytics, no stranger to enterprise data-analysis needs.</li> </ul></li> <li><strong>Python's new software repository system could lead to enterprise-friendly Python package management</strong> <ul> <li>One possibility being floated in this vein is the concept of an enterprise-grade package index for Python, as <a href="">discussed by Cristian Medina of Nimble Storage</a>:</li> <li>Businesses always have a need for an on-premises, secure, encrypted and highly available distribution mechanism of compiled binaries. Together with setuptools providing various install capabilities that can cover non-Python code just as well, it seems like we could put together a decent product. Something like a Docker Trusted Registry.</li> <li>I’m working on something here too that come out of the discussion of Talk Python #84</li> </ul></li> </ol> <p><strong>#3 (Brian) <a href="">From Python to Numpy</a></strong></p> <ul> <li>Nicolas P. Rougier -</li> <li>“Nicolas P. Rougier is a full-time research scientist at Inria which is the French national institute for research in computer science and control.”</li> <li>Creative commons book for people who are intermediate level Python developers and want to use numpy for science or engineering.</li> </ul> <p><strong>#4 (Michael):</strong> <a href=""><strong>openai/universe</strong></a></p> <ul> <li>Universe: a software platform for measuring and training an AI's general intelligence across the world's supply of games, websites and other applications. <a href=""></a></li> <li>Universe allows anyone to train and evaluate AI agents on an extremely wide range of real-time, complex environments.</li> <li>Universe makes it possible for any existing program to become an OpenAI Gym environment, without needing special access to the program's internals, source code, or APIs. <ul> <li>It does this by packaging the program into a Docker container</li> <li>presenting the AI with the same interface a human uses: </li> <li>sending keyboard</li> <li>mouse events</li> <li>receiving screen pixels</li> <li>Our initial release contains over 1,000 environments in which an AI agent can take actions and gather observations.</li> </ul></li> </ul> <p><strong>#5: (Brian): <a href="">Python requests deep dive</a></strong></p> <ul> <li>Anthony Shaw - on</li> <li>Converted a large project from httplib to requests</li> <li>Apache Libcloud <ul> <li>needed to provide a set of base classes that would handle HTTP and HTTPS REST/JSON, REST/XML and various other bizarre HTTP APIs. </li> <li>Libcloud has over 80 client libraries for every major cloud service out there. </li> <li>single Connection class that handles encoding and decoding of JSON, XML or Raw data.</li> </ul></li> <li>Anthony walks through the types of changes made, including authentication, session handling, testing, prepared requests, streams, … </li> <li>uses <a href="">requests-mock</a> <ul> <li>very cool API</li> </ul></li> </ul> <p><strong>#6 (Michael):</strong> <a href="">What's the community favorite for developing OSX desktop applications with Python?</a></p> <ul> <li><a href=""><strong>bangeron</strong></a><strong>:</strong> I've been using PyQt for all my projects with a GUI: - It has a native look and feel - It's cross-platform (I do everything on macOS) - It's a whole framework, including modules for plotting charts, networking, SQL, etc. - It has <em>excellent</em> documentation. - It has a GUI editor (Qt Creator) that is pretty good, once you figure it out.</li> <li><a href=""><strong>Omnius</strong></a>: Same but using pyside to avoid license fees.</li> <li><a href=""><strong>spinwizard69</strong></a>: Use TK for simple apps! It is included with Python and avoids dependency hell.</li> <li><a href=""><strong>EssaAlshammri</strong></a>: Take a look at <a href="">Toga</a> its idea is great.</li> <li><a href=""><strong>Ruditorres</strong></a>: Kivy hands down, it is pythonic, easy, and beautiful</li> <li><a href=""><strong>bangeron</strong></a>: Kivy is anything but beautiful. At least, not without spending some time creating your own theme.</li> <li>Personally, I want py_electron (modeled after <a href="">electronjs</a>).</li> <li>Notable mention, related: <a href="">cx_Freeze released</a> (Version 5.0.1 (January 2017))</li> </ul>
Jan 17, 2017
#8 Python gets Grumpy, avoiding burnout, Postman for API testing and more
<p>This is Python Bytes, Python headlines and news deliver directly to your earbuds: episode 8, recorded on <strong>January 10th, 2017</strong>. In this episode we discuss <strong>Python is Grumpy, avoiding burnout, Postman for API testing and more</strong>. </p> <p><strong>#1 (Brian): <a href="">Jessica McKellar, &quot;Breaking The Rules&quot;, PyBay2016</a><a href=""> </a></strong></p> <ul> <li>Jessica <ul> <li>was directory of PSF for several years, was involved with Boston Python UG, is diversity chair for PyCon, is an engineering director at dropbox</li> </ul></li> <li>Powerful keynote <ul> <li>Her extra work with Python is not about Python, it’s about studying systems.</li> <li>“Learning how to program changes the way you think about, debug, and interact with the world”</li> <li>“You learn a set of rules to build software, … then you learn that you can change the rules.”</li> <li>“Programmers master a system they know they can change.”</li> <li>“This comes from the tenets of free software.”</li> <li>“We take for granted that changing something to make it better is just a thing you do when you need to.”</li> </ul></li> <li>This can and should carry over to the rest of your life.</li> <li>Jessica takes this idea and applies it to politics, voting, and polling stations, and ran a polling station herself.</li> <li>That’s pretty incredible. About half the time is Q&amp;A with some great questions.</li> <li>Listen to this talk and apply it to every part of your work and life.</li> </ul> <p><strong>#2 (Michael): <a href="">Grumpy is a Python to Go</a></strong></p> <ul> <li>By Dylan Trotter from YouTube</li> <li>Grumpy is a Python to Go source code transcompiler and runtime.</li> <li>intended to be a near drop in replacement for CPython 2.7</li> <li>The key difference is that it compiles Python source code to Go source code which is then compiled to native code, rather than to bytecode. This means that Grumpy has no VM.</li> <li>6,000 stars on Github in 3 weeks</li> <li>Look for him on Talk Python To Me (episode 95?)</li> </ul> <p><strong>#3 (Brian): <a href="">Finding dead code with Vulture - Dougal Mathews</a></strong></p> <ul> <li><code>pip install vulture</code> , then <code>vulture some/directory/of/code</code> </li> <li>Reports unused code.</li> <li>vs You can get similar information from coverage if your test suite or the code you run during the coverage inspection is fairly complete. However, what if a unit test is the only thing calling some function? vulture allows you to exclude your test code when looking for unused code.</li> <li>vs static analyzers like flake8. With some of my own code that I have in progress, vulture found the same stuff that flake8 did. However, if you are only looking for dead code, it’s easier to find with vulture if you have other flake8 violations. Also some folks don’t like style checkers.</li> <li>I’d like to hear what other people think. But I like the idea of having a focused dead code tool. And vulture is a great name for such a tool.</li> </ul> <p><strong>#4 (Michael): <a href="">Postman: Developing APIs is hard. Postman makes it easy</a></strong></p> <ul> <li>A powerful GUI platform to make your API development faster &amp; easier, from building API requests through testing, documentation and sharing.</li> <li>Cross platform and free</li> <li>A simple and effective way to share details about your public-facing API </li> <li>Testing <ul> <li>Run Postman Collections directly from the command line</li> <li>integrating with continuous integration servers and builds</li> <li>Monitor uptime, performance and correctness of your APIs.</li> </ul></li> <li>Notable mention goes to <a href=""></a> (macOS only)</li> </ul> <p><strong>#5 (Brian): <a href="">The Reality of Developer Burnout</a><a href=""> by </a><a href="">Kenneth Reitz</a></strong></p> <ul> <li>Author of <a href="">Requests</a> and of <a href="">Maya (datetime for humans)</a>, covered in the <a href="">last episode</a>.</li> <li>This is an article about getting overwhelmed as a maintainer of an open source project. But applies to anyone supporting a tool, even for your coworkers. And really applicable to all sorts of developer burnout. </li> <li>Advice: <ul> <li>Keep producing, but stop consuming so much on social networks. twitter, reddit, etc.</li> <li>Delegate more.</li> <li>Have hobbies other than coding</li> </ul></li> <li>side note: <ul> <li>On the maya github page, I noticed a link <a href=""></a>, a Kenneth project where you can set up a way for people to just send you a thank-you note. I think this is cool. I wrote about <a href="">the power of “thank you”</a> a few years ago. It’s really important in open source, and really all the time. </li> </ul></li> </ul> <p><strong>#6 (Michael): <a href="">Jinja 2.9 Released by Armin Ronacher</a></strong></p> <ul> <li>From Hugh Blandford (thanks!)</li> <li>New release, 2.9 codename "Derivation" of the Jinja template engine for Python is out</li> <li>While Jinja2 supported Python 3 for years now it never really embraced functionality that the language provides on 3.x that it does not do on 2.x <ul> <li>However 3.6 now added async generators which permits Jinja2 to fully support the async and await keywords on 3.6 and later.</li> <li>In particular it means that you can now return coroutines from functions passed to Jinja2 templates and the template engine will automatically await them.</li> <li>Likewise all filters were updated to work with iterators as well as async iterators alike.</li> </ul></li> </ul>
Jan 10, 2017
#7 Python 3.6 is out, Sanic is a blazing web framework, and are failing our open source infrastructure?
<p>This is Python Bytes, Python headlines and news deliver directly to your earbuds: episode 7, recorded on Wednesday, January 4th. </p> <p>In this episode we discuss Python 3.6 being release, a blazing Python web framework called Sanic, how we are failing our open source infrastructure, and more. </p> <p>This episode was brought to you by <a href="">Rollbar: they help you take the pain out of errors</a>.</p> <p><strong>#1 (Brian)</strong>: <strong>Python 3.6 is officially released</strong></p> <ul> <li>Dec 23 Announcement on <a href=""></a></li> <li>What’s new article on <a href=""></a></li> </ul> <p>Here’s a few more articles:</p> <ul> <li>Python 3.6 is packed with goodness - Serdar Yegulalp - InfoWorld <ul> <li><a href=""></a></li> <li>A list of cool things in Python 3.6</li> <li>async/await in more places</li> <li>Improved memory usage and speed</li> <li>frame evaluation API to provide better support for JITs, tracers, debuggers</li> <li>secrets module for common security-related functions such as generating tokens</li> </ul></li> <li>One thing that could stop a team from switching to Python 3 would be that they looked into it once, and a package they needed was not ported yet. That’s different now, most are ported, or have been replaced with some other way.</li> <li>Adopt Python 3 - Dibya Chakravorty - on Medium <ul> <li><a href=""></a> </li> <li>A discussion of packages that support Python 2 and 3, those that are 2 only, those that are 3 only.</li> <li>Two lists of most downloaded packages and their py3 readiness:</li> <li>Wall of superpowers: <a href=""></a>, 187/200 = 93.5%</li> <li>Python 3 readiness: <a href=""></a>, 341/360 = 94.7%</li> <li>Go further:</li> <li>all pypi packages that are stable and active (at least one release in 2016), about 6000 packages</li> <li>total python 3 coverage is 72%</li> <li>14% are Python 3 only</li> <li>28% are Python 2 only</li> <li>of the 28%, author estimates only 25% of the remaining packages would be difficult to port, based on their size, and included that list.</li> <li>Also entire code for this article is in an iPython notebook:<a href=""></a></li> </ul></li> </ul> <p><strong>#2 (Michael)</strong> <strong>Roads and Bridges: The Unseen Labor Behind Our Digital Infrastructure by Nadia Eghbal</strong> <a href=""></a></p> <ul> <li>From The Ford Foundation, 14 JULY 2016</li> <li>Features Eric Holscher from Read the Docs &amp; Donald Stufft from PyPI</li> <li>Discussed on Talk Python episode <a href="">#84: Are we failing to fund Python's core infrastructure?</a></li> <li>Some highlights from the 149 page report: <ul> <li>Our modern society—everything from hospitals to stock markets to newspapers to social media—runs on software. But take a closer look, and you’ll find that the tools we use to build software are buckling under demand. (roads-and-bridges-the-unseen-labor-behind-our-digital-infrastructure, p.4)</li> <li>No individual company or organization is incentivized to address the problem alone, because open source code is a public good. (roads-and-bridges-the-unseen-labor-behind-our-digital-infrastructure, p.9)</li> <li>By 2014, two-thirds of all Web servers were using OpenSSL, enabling websites to securely pass credit card and other sensitive information over the Internet. (roads-and-bridges-the-unseen-labor-behind-our-digital-infrastructure, p.11)</li> <li>Institutional efforts to support digital infrastructure: There are some institutional efforts to collectively organize and help support open source projects. Some are independent software foundations; other sources of support come from software companies themselves. Administrative support and fiscal sponsorship Several foundations provide organizational support, such as fiscal sponsorship, to open source projects: in other words, taking care of the non-coding tasks that many developers would prefer not to do (roads-and-bridges-the-unseen-labor-behind-our-digital-infrastructure, p.109)</li> <li>Of Heartbleed, Marquess wrote, “The mystery is not that a few over-worked volunteers missed this bug; the mystery is why it hasn’t happened more often.” (roads-and-bridges-the-unseen-labor-behind-our-digital-infrastructure, p.13)</li> <li>Heartbleed, had been included in a 2011 update. It had gone unnoticed for years. Heartbleed could allow any sophisticated hacker to capture secure information being passed to vulnerable web servers, including passwords, credit card information, and other sensitive data (roads-and-bridges-the-unseen-labor-behind-our-digital-infrastructure, p.13)</li> <li>People expressed their support by sending donations to the foundation. Although Marquess was grateful for their enthusiasm, the first round of donations came out to roughly $9,000: not nearly enough to sustain a team. (roads-and-bridges-the-unseen-labor-behind-our-digital-infrastructure, p.13)</li> <li>After Heartbleed, OpenSSL finally got more of the funding it needed—at least for now. They currently have enough money to pay four full-time employees for three years. But a year and a half into that funding, Marquess isn’t sure what will come next. (roads-and-bridges-the-unseen-labor-behind-our-digital-infrastructure, p.14)</li> <li>Free software makes it exponentially cheaper and easier to build software (roads-and-bridges-the-unseen-labor-behind-our-digital-infrastructure, p.23)</li> <li>Free software is directly responsible for today’s current startup renaissance (roads-and-bridges-the-unseen-labor-behind-our-digital-infrastructure, p.24)</li> <li>Software not getting the necessary maintenance it needs Building digital infrastructure in a haphazard fashion means that all software gets built more slowly and inefficiently. One example of this can be found in the history of Python infrastructure. An important infrastructure project for Python developers is called setuptools. (roads-and-bridges-the-unseen-labor-behind-our-digital-infrastructure, p.80)</li> </ul></li> </ul> <p><strong>#3 (Brian)</strong>: <strong>Matplotlib 2.0.0 rc2 was released by Thomas Caswell</strong></p> <ul> <li>install instructions: <a href=""></a></li> <li>List of top level changes in release history on github -<a href=""></a></li> <li>Changes to the default style -</li> <li><a href=""></a></li> <li>Post visually shows all of the new changes. </li> <li>What’s new in matplotlib 2.0 - also on <a href=""></a></li> <li><a href=""></a></li> </ul> <p><strong>#4 (Michael)</strong> <strong>Introduction to MongoDB and Python</strong> <a href=""></a></p> <ul> <li>SQL vs NoSQL</li> <li>PyMongo <ul> <li>Inserting and querying data</li> </ul></li> <li>MongoEngine <ul> <li>Classes</li> <li>Constraints </li> <li>OOP</li> </ul></li> <li>Michael’s 1.5 hour presentation at Software Architect conference in London: <a href="">Applied NoSQL with MongoDB and Python</a></li> </ul> <p><strong>#5: (Brian)</strong>: <strong>Introducing Maya: Datetimes for Humans</strong> <a href=""></a> - Working with dates is harder than it should be. Kenneth has proven that he understands how to make reasonable interfaces to hide complicated things.</p> <p>From "why is this useful?" section</p> <ul> <li>All timezone algebra will behave identically on all machines, <em>regardless of system locale</em>.</li> <li>Complete symmetric import and export of both <strong>ISO 8601</strong> and <strong>RFC 2822</strong> datetime stamps.</li> <li>Fantastic parsing of both dates written for/by humans and machines (<strong>maya.when()</strong> <em>vs.</em> <strong>maya.parse()</strong>).</li> <li>Support for human slang, both import and export (e.g. 'an hour ago').</li> <li>Datetimes can very easily be generated, with our without timezone information attached (naive).</li> <li>This library is based around epoch time, but dates before Jan 1 1970 are indeed supported, via negative integers.</li> </ul> <p><strong>#6 (Michael)</strong>: <strong>Sanic: Python 3.5+ web server that's written to go fast</strong> <a href=""></a></p> <ul> <li>Sanic is a Flask-like Python 3.5+ web server that's written to go fast. It's based on the work done by the amazing folks at magicstack, and was inspired by this article: <a href=""></a>.</li> <li>On top of being Flask-like, Sanic supports async request handlers. This means you can use the new shiny async/await syntax from Python 3.5, making your code non-blocking and speedy.</li> </ul> <p>A basic action method</p> <pre><code>@app.route("/") async def test(request): # async def !!! return json({ "hello": "world" }) </code></pre> <p><strong>Performance numbers</strong></p> <table> <thead> <tr> <th>Server</th> <th>Implementation</th> <th>Requests/sec</th> <th>Avg Latency</th> </tr> </thead> <tbody> <tr> <td>Sanic</td> <td>Python 3.5 + uvloop</td> <td>33,342</td> <td>2.96ms</td> </tr> <tr> <td>Bottle</td> <td>gunicorn + meinheld</td> <td>13,596</td> <td>7.36ms</td> </tr> <tr> <td>Flask</td> <td>gunicorn + meinheld</td> <td>4,988</td> <td>20.08ms</td> </tr> <tr> <td>Aiohttp</td> <td>Python 3.5 + uvloop</td> <td>2,979</td> <td>33.42ms</td> </tr> <tr> <td>Tornado</td> <td>Python 3.5</td> <td>2,138</td> <td>46.66ms</td> </tr> </tbody> </table>
Jan 04, 2017
#6 Python 3.6 is going to be awesome, Kite: your friendly co-developing AI
<p>This is Python Bytes, Python headlines and news deliver directly to your earbuds: episode 6, recorded on Monday, December 12th. In this episode we discuss why Python 3.6 is going to be awesome, kite: your friendly co-developing AI, and more! </p> <p>This episode was brought to you by <a href="">Rollbar: they help you take the pain out of errors</a>.</p> <p>This is the last episode of 2016. Thank you everyone for a great launch. We’ll be back early January. 😉 Be sure to check out Talk Python and Test and Code if you want more Pythonic listening over the break.</p> <h2>News items</h2> <p><a href=""><strong>#1 Make your Python code more readable with custom exception classes</strong></a></p> <ul> <li>This is a 5 min video + text. Good introduction into why you should define your own exceptions instead of using the built in ones, and how to do it. <br /> <ul> <li>It makes errors from your code more readable.</li> <li>Better communication between your code and the person using your code.</li> <li>It allows you to give more context of the error to the caller of the function.</li> <li>Remember to derive from Exception or from another builtin exception.</li> </ul></li> <li>Do people create enough fine-grained exception types? I would say probably not.</li> <li>This advice is good because it encourages <a href="">EAFP</a> (easier to ask for forgiveness than permission) style of programming which is generally Pythonic.</li> <li>Allows for multiple except statements for different errors in one try block</li> <li>Dan also featured our show in <a href="">The ultimate list of Python Podcasts</a> (thanks Dan!)</li> <li>If you have a package that defines it’s own exceptions, please read another article. <ul> <li><strong>The definitive guide to Python exceptions</strong> <ul> <li>Julien Danjou</li> <li></li> <li>Covers having a common base exception for your package, organization within a package, and some examples of packages that organize their exceptions well, including requests</li> </ul></li> </ul></li> </ul> <p><a href=""><strong>#2 Kite</strong></a></p> <ul> <li>Kite augments your coding environment with all the internet’s programming knowledge.</li> <li>Is an AI pair programmer, or mentor really.</li> <li>Contextual info for <ul> <li>language</li> <li>packages</li> <li>e.g. “import r” → shows list of popular packages</li> <li>then detailed docs, examples, etc.</li> <li>autocompletions… by global popularity</li> <li>BYOE</li> <li>even works on your code</li> <li>be sure to watch the video</li> </ul></li> <li><a href=""></a> is implemented mostly in Go <a href="">according to the founder</a> Adam Smith.</li> <li>Thanks Gilberto Diaz for sending this one to us.</li> </ul> <p><a href=""><strong>#3 Tidy Data in Python</strong></a> (by Jean-Nicholas Hould)</p> <ul> <li>This article caught my attention because of the notion that the data as you receive it might not be in a form that is ideal to use it. This I am used to. But the article give some attributes of what problems to look for in data sets, and how to transform the data into a more usable structure using pandas.</li> <li>Great example of someone taking a good idea from someone else, summarizing it, and showing how to use it in Python.</li> <li>Based on a paper named <a href="">Tidy Data</a> by Hadley Wickham <ul> <li>In this post, I will summarize some tidying examples Wickham uses in his paper </li> <li>Will demonstrate how to do so using the Python pandas library</li> </ul></li> <li>Tidy data has the following attributes: <ul> <li>Each variable forms a column and contains values</li> <li>Each observation forms a row</li> <li>Each type of observational unit forms a table</li> </ul></li> <li>A few definitions: <ul> <li>Variable: A measurement or an attribute. Height, weight, sex, etc.</li> <li>Value: The actual measurement or attribute. 152 cm, 80 kg, female, etc.</li> <li>Observation: All values measure on the same unit. Each person.</li> </ul></li> </ul> <p><a href=""><strong>#4 What's new in Python 3.6</strong></a></p> <ul> <li>By Brett Cannon <ul> <li>Works at Microsoft Azure Data Science team</li> <li>Python core developer</li> </ul></li> <li>16 PEPs in Py3.6 - more than any other release than Py 3.0</li> <li>PEP 498 Formatted string literals <ul> <li>You learn about internals</li> <li>That this is actually faster than str.format() because optimizations that can be done on the string itself (f””)</li> <li>PEP 524: On Py 3.5 would fall back to unsecure. On Py 3.6 os.urandom() now blocks os.urandom() for cryptographically strong random numbers or os.getrandom() raises error if not enough randomness. Usually not a problem, but with things like containers and IoT, it has become one! Fix: use new secrets module.</li> </ul></li> <li>There are also other interesting things that aren’t PEPs </li> <li>Python 3.6 is generally significantly faster (than Py3.5 and legacy Python)</li> <li>Python 3.6.0 release candidate is now available, final expected end of the week</li> <li>Something that hasn’t been as highly publicized is the deprecation of pyvenv. <ul> <li></li> <li>“The <code>pyvenv</code> script has been deprecated as of Python 3.6 in favor of using <code>python3 -m venv</code> to help prevent any potential confusion as to which Python interpreter a virtual environment will be based on.”</li> <li>This is important for me and you and anyone who teaches people to use Python. We often recommend virtual environments, and it’s good to recommend -m venv to make sure people know which Python interpreter they are tying to their virtual environment.</li> </ul></li> </ul> <p><a href=""><strong>#5: Who Tests What</strong></a></p> <ul> <li>I had <a href="">Ned Batchelder on Test&amp;Code to discuss coverage</a>. Episode 12.</li> <li>Ned is planning a new feature for <a href=""></a>, which would tell you not just which code has run, but which test (or something else) caused that code to run.</li> <li>Discusses the stages of coverage, measurement/storing data/combining/reporting. Discusses the issue of how to decide who is calling the code in question. His current model is based on coverage having a plugin hook for someone to tell coverage a string that defines the “what” that is causing the code to be run. He also discusses some decisions about storage concerns and what coverage does now.</li> <li>He has questions remaining that he wants help with: <ul> <li>Today keeps everything in memory until the end of the process, then writes it all to disk. <strong>Q:</strong> Will we need something more sophisticated? Can we punt on that problem until later?</li> <li><strong>Q:</strong> Is it important to try to conserve memory?</li> <li>Today, the .coverage data files are basically JSON. This much data might need a different format. <strong>Q:</strong> Is it time for a SQLite data file?</li> <li><strong>Q:</strong> How would you use the data?</li> <li>And a couple more questions regarding reporting.</li> </ul></li> <li>I like the direction this is going and I encourage everyone who has some nonstandard usage of coverage to take a look at this and give Ned some feedback.</li> </ul> <p><a href=""><strong>#6 Threaded Asynchronous Magic and How to Wield It</strong></a> (by <a href="">Cristian Medina</a>)</p> <ul> <li>This is your async programming in Python 3.5+ via async / await article</li> <li>Covers: <ul> <li>Tasks</li> <li>Scheduling tasks</li> <li>Scatter / gather example</li> <li>Moving the asyncio loop to a background thread</li> </ul></li> <li>Examples: <ul> <li>Real World Example #1 — Sending Notifications (email)</li> <li>Real World Example #2 — Parallel Web Requests <ul> <li>via <a href="">aiohttp</a>: HTTP client/server for asyncio (PEP 3156)</li> </ul></li> </ul></li> </ul> <p><strong>Update: From episode 3: pynini</strong> <a href="">āṇini</a></p>
Dec 12, 2016
#5 Legacy Python vs Python and why words matter and Request's 5 Whys retrospective
<p>This is Python Bytes, Python headlines and news deliver directly to your earbuds: episode 5, recorded on December 5, 2016. In this episode we discuss Legacy Python vs Python and why words matter and Request's 5 Whys retrospective. </p> <p>This episode was brought to you by <a href="">Rollbar: they help you take the pain out of errors</a>.</p> <h2>News items</h2> <p><strong>#1 The Five Why’s of Requests 2.12</strong> <a href=""></a></p> <ul> <li>Some decision we made breaks a whole bunch of people’s code, and then each fix we attempt to make produces new, unforseen problems. These releases can be very dispiriting for project maintainers, because they make it seem like every time we try to improve something we get stuck on the mistakes our past selves made.</li> <li>Requests has had a few of these in the past, and right now we appear to be in the middle of another one. So far the 2.12 release series has caused problems for a number of users</li> <li>Problem: The current release of Requests, v2.12.2, breaks anyone who uses a URL with a scheme that isn’t “http” or “https” and who passes the params keyword to a Requests URL.</li> <li>Ultimately, our current issue comes from the fact that we have been inconsistent on our stance regarding URLs with weird schemes</li> </ul> <p><strong>#2 Planning an Early Death for Python 2</strong> <a href=""></a> via <a href=""></a></p> <ul> <li>From Data Structure for Data Science workshop gathered at UC Berkeley's BIDS [Berkeley Institute for Data Science]</li> <li>Out of the discussion arose a topic that has long plagued the python community at large: code that requires legacy Python 2.7 is holding back the development of data-science toolsets and – by extension – the progress of data science as a whole. Python 2.7 was an important part of the history of scientific computing, but now it should be left as part of that history. Thus, we convened a small working group to plan an early death for Legacy Python.</li> <li>So what are the step we can do to push the transition forward?</li> <li>Choose your words: <ul> <li>Assume that Python 3 is just Python, and refer to Python 2 as legacy python.</li> <li>Refer to Legacy Python in the past tense (reinforce the old and deprecated state of Legacy Python)</li> </ul></li> <li>Make your examples/documentation Python 3 only <ul> <li>Sprinkle with function annotation, and async/await keyword can help with communicating your example are Python 3 only…</li> </ul></li> <li>Ask user to reproduce but on up-to-date Python version</li> <li>Testing: Make sure that all the project you care about have continuous integration on Python 3</li> </ul> <p><strong>#3 Simplifying complex business logic with Python's Kanren</strong> <a href=""></a></p> <p><strong>#4 If Reddit were written from scratch today, which Python web framework would it use and why?</strong> <a href=""></a> </p> <ul> <li>Today (from SQLAlchemy's page): Reddit is one of the biggest social news aggregators on the internet. Reddit is built using Pylons, Mako templates, and a custom, non-ORM database abstraction layer built on SQLAlchemy Core.</li> <li>Ergo14: I would say that the most sane option would be Pyramid. It is faster than Django in tests and it doesn't repeat mistakes with threadlocals that flask or pylons did in past. I did some both small and medium-big (20+ million user applications) and it just feels right. It doesn't try to get in your way and give you magic solutions to your problems.</li> <li>Wting: I'm assuming you're talking about reddit at its current scale. Not Flask. Too many global variables and it's not thread safe (for async). Not Django. It's too opinionated and everything is in-housed for scaling reasons. My guess is Pyramid (the successor to Pylons). In fact, that's what reddit's current and future services are built with.</li> <li>So, Python web frameworks tend to cause a very strong, very subjective split in opinion.</li> </ul> <p><strong>#5: Two testing related articles</strong></p> <ul> <li><strong>Getting started with pytest</strong> <a href=""></a> <ul> <li>Anything that promotes pytest without presenting wrong information is good</li> </ul></li> <li><p><strong>The Best New Feature in unittest You Didn’t Know You Need</strong> <a href=""></a></p> <ul> <li>First article I’ve seen explaining subtest</li> <li>Subtest was added in Python 3.4</li> </ul></li> </ul> <p><strong>#6 PyTone</strong> <a href=""></a> PyTone is a music jukebox written in Python with a curses based GUI. While providing advanced features like crossfading and multiple players, special emphasis is put on ease of use, turning PyTone into an ideal jukebox system for use at parties.</p> <p>Thx @kidpixo: <a href=""></a></p>
Dec 05, 2016
#4 Python 3 is just fine for beginners thank you, q is awesome for debugging, and more
<p>This is Python Bytes, Python headlines and news deliver directly to your earbuds: episode 4: recorded on November 28, 2016. In this episode we cover the case for Python 3, asyncio, pyston,, and q. </p> <p>This episode was brought to you by <a href="">Rollbar: they help you take the pain out of errors</a>.</p> <h2>News items</h2> <p><strong>The Case FOR Python 3</strong></p> <pre><code>&gt;&gt; "It's as simple as that. If you learn Python 2, then you can still work with all the legacy Python 2 code in existence **until Python dies or you (hopefully) move on**. But if you learn Python 3 then your future is very uncertain. You could really be learning a dead language and end up having to learn Python 2 anyway." -- Zed S. </code></pre> <ul> <li>This is harmful to the Python community.</li> <li>Let's stop recommending this book and stop elevating this guy's work</li> <li>"Arguments" from Zed: <ul> <li>Not In Your Best Interests: "The Python project’s efforts to convince you to start with Python 3 are not in your best interest, but, rather, are only in the best interests of the Python project."</li> <li>You Should Be Able to Run 2 and 3 (in the same process). The fact that no one has (yet) written a Python 3 interpreter that can simultaneously run Python 2 in the same process means Python 3 is not Turing complete</li> <li>Difficult To Use Strings: "The strings in Python 3 are very difficult to use for beginners. In an attempt to make their strings more “international” they turned them into difficult to use types with poor error messages."</li> <li>Final analysis: <ul> <li>You can’t add b"hello" to "hello".</li> <li>Too many formatting choices for strings</li> <li>Bytes aren't automatically decoded to strings</li> </ul></li> <li>Therefore: All newbies should avoid Python 3 like the plague - it will curse your career and cloud your judgement.</li> </ul></li> <li>Excellent rebuttal article: <a href=""><strong>Rebuttal for Python 3</strong></a></li> </ul> <p><a href=";sh=34a66cab"><strong>Reddit discussion of a Feb article on &quot;AsyncIO for the Working Python Developer&quot;</strong></a> - Article: <a href="">AsyncIO for the Working Python Developer</a></p> <ul> <li>Article is a nice introduction, but uses Python 3.4 syntax</li> <li>Interesting to note that no one in the reddit comments suggests a better 3.5 tutorial. Is async/await just too confusing for a 3.5 intro article? Or is it so simple that no one has thought it needs a tutorial. </li> <li>23 page article / tutorial on Python 3.4's asyncio</li> <li>Not covering Python 3.5's async / await, but great gateway article</li> <li>Here’s a call for action. Someone write a similar article but with 3.5+ syntax. Also could we show examples without sleep calls. Sleep is such a lame way to show “do some actual work”. Maybe use a producer/consumer or reader/writer example or something.</li> </ul> <p><a href=""><strong>Pyston 0.6 released</strong></a></p> <ul> <li>Pyston is an open source Python implementation that aims to be both highly compatible and high-performance. </li> <li>It uses modern JIT techniques and natively supports many CPython C extension modules. </li> <li>Pyston is sponsored by Dropbox, and is pronounced "piston".</li> <li>main goal was to reduce the overall memory footprint. It also contains a lot of additional smaller changes for improving compatibility and fixing bugs. <ul> <li>We also spent time on making it easier to replace CPython with Pyston, such as by more closely matching its directory structure </li> <li>following its ‘dict’ ordering. </li> <li>We can now, for example, run pip and virtualenv unmodified, without requiring any upstream patches like other implementations do.</li> </ul></li> <li>Unclear how much of a research project vs an actual project this is at the moment</li> </ul> <p><a href=""><strong>Announcing</strong></a></p> <ul> <li>Automatic API documentation from pypi repositories.</li> <li> will eventually auto-generate API reference documentation for every package on PyPI.</li> </ul> <p><a href=""><strong>What one thing took your Python to the next level</strong></a></p> <ol> <li>Mastering generators</li> <li>Understanding how iteration really works in Python opens up many possibilities for elegant, high performance code. Use iPython. Learn that for does tuple unpacking; play with zip, enumerate, all, any; take a look at the itertools module</li> <li>Unit tests! Pytest in particular! Gives you a whole new perspective on your code and dammit, it's so satisfying to get all those green OKs!</li> <li>list comprehensions</li> <li>For me, all the work of David Beazley, in particular <a href=""></a></li> </ol> <p><a href=""><strong>q : Quick-and-dirty debugging output for tired programmers</strong></a></p> <ul> <li>Learned about it from a tweet from Luciano Ramalho.</li> <li><a href=""></a></li> <li>5 min lightning talk: <a href=""></a> </li> <li>q on pypi: <a href=""></a></li> </ul> <h2>Our personal news</h2> <h3>Michael</h3> <ul> <li>Follow up: <a href=""></a> Python 3.6 Release Schedule Final date: 2016-12-16</li> </ul> <h3>Brian</h3> <ul> <li>Hard at work on pytest book that is a followup to current <a href="">Python Testing book</a>.</li> </ul>
Nov 29, 2016
#3 Python 3.6 is coming, and it's awesome plus superior text processing with Pynini
<p>Welcome to Python Bytes. Python headlines delivered directly to your earbuds. In this episode we cover the new features in Python 3.6, text processing with Pynini, Python is 2nd most popular language on GitHub and more. </p> <p>This episode was brought to you by <a href="">Rollbar: they help you take the pain out of errors</a>.</p> <h2>News items</h2> <p><a href=""><strong>How to get superior text processing with Pynini</strong></a></p> <ul> <li>how to change your thinking from regular expression matching to FSAs (finite-state acceptor), then FSTs.</li> <li>Example using cheese names (love the Python tradition with that).</li> <li>library developed at Google called pynini for easily using FSTs</li> </ul> <p><a href=""><strong>Python 3.6 b4 Is out! (beta!!!)</strong></a></p> <ul> <li><a href="">PEP 498</a>, formatted string literals.</li> <li><a href="">PEP 515</a>, underscores in numeric literals.</li> <li><a href="">PEP 525</a>, asynchronous generators.</li> <li><a href="">PEP 530</a>: asynchronous comprehensions.</li> <li>The <strong>dict</strong> type has been reimplemented to use a faster, more compact representation similar to the PyPy dict implementation. This resulted in dictionaries using 20% to 25% less memory when compared to Python 3.5.</li> <li>The class attribute definition order is now preserved.</li> <li><strong>Security</strong>: The new secrets module has been added to simplify the generation of cryptographically strong pseudo-random numbers suitable for managing secrets such as account authentication, tokens, and similar.</li> <li><strong>Windows</strong>: python.exe and pythonw.exe have been marked as long-path aware, which means that when the 260 character path limit may no longer apply. See removing the MAX_PATH limitation for details.</li> </ul> <p><a href=""><strong>Create an Excellent Python Dev Env</strong></a></p> <ul> <li>pyenv, like virtualenv, I think, but independent of Python.</li> <li>pyenv-virtualenv instead of virtualenvwrapper</li> <li>pipsi - a tool to let you install python based CLI utilities in their own virtualenv easily. </li> <li>Tools he installs with pipsi: <ul> <li>tox - testing multiple environments</li> <li>mkdocs - documentation static site generator</li> <li>git-review - gerrit integration with git. </li> <li>flake8 - static analysis</li> </ul></li> <li>He also mentions a shell called fish, which I hadn’t heard of before.</li> </ul> <p><a href=""><strong>GitHub language statistics, Python is 2nd most popular language</strong></a></p> <ul> <li>An active repository should meet this requirements: created or updated during the evaluated period, <ul> <li>at least one star</li> <li>at least one fork</li> <li>a size larger than 10 Kb</li> </ul></li> <li>JavaScript is likely over counted</li> </ul> <p><a href=""><strong>Handling Unicode Strings in Python</strong></a></p> <ul> <li>Text Representation in Python and the differences between 2.7 and 3.4+</li> <li>Converting between unicode strings and bytes</li> <li>IO boundary issues with databases, file system, or network services</li> <li>Logging</li> <li>JSON encoding</li> <li>Redis</li> <li>A pointer to a 2012 article from Ned Batchelder called Pragmatic Unicode.</li> </ul> <p><a href=""><strong>Python extensions for VS Code for Mac / PC / Linux</strong></a></p> <ul> <li>Linting (Prospector, Pylint, pycodestyle/Pep8, Flake8, pylama, pydocstyle with config files and plugins)</li> <li>Intellisense (autocompletion with support for PEP-0484)</li> <li>Scientific tools (Jupyter/IPython)</li> <li>Auto indenting</li> <li>Code formatting (autopep8, yapf, with config files)</li> <li>Code refactoring (Rename, Extract Variable, Extract Method, Sort Imports)</li> <li>Viewing references, code navigation, view signature</li> <li>Excellent debugging support (remote debugging over SSH, mutliple threads, django, flask)</li> <li>Unit testing, including debugging (unittest, pytest, nosetests, with config files)</li> <li>Execute file or code in a python terminal</li> <li>Local help file (offline documentation)</li> <li>Snippets</li> </ul> <h2>Our personal news</h2> <h3>Brian</h3> <ul> <li>Lot’s of writing. My target is to get the book in print by PyCon US 2017 in May. But that’s an aggressive schedule with the rest of my projects.</li> <li>Still working on getting #25 out the door. Interview with Dave Hunt from Mozilla about Selenium, Mozilla, pytest, tox, CI, and much more. Should go out within the next couple of days. </li> <li>Next interview scheduled is with David Hussman from DevJam. This will be a higher level discussion about software development practices. Look for that this month also.</li> </ul> <h3>Michael</h3> <ul> <li>Shipped: <a href="">Episode #85: Parsing horrible things with Python</a></li> <li>Quick correction: github's awesome-python isn't the newsletter, that's <a href=""></a></li> </ul>
Nov 23, 2016
#2 PyCon, awesome python, python developer job prospects, and more
<p>Welcome to Python Bytes. Python headlines delivered directly to your earbuds. In this second episode we cover a couple of key python package releases, pycon, awesome python, python developer job prospects, and more! </p> <h2>News items</h2> <p><a href=""><strong>Celery 4.0</strong></a></p> <ul> <li>A task queue and scheduling subsystem</li> <li>Supposedly a massive release with over 2 years of changes</li> <li>Will be the last major release with Python 2.x support, already dropping 2.6</li> </ul> <p><a href=""><strong>Psutil 5.0.0</strong></a></p> <ul> <li>Used for process monitoring, system utilization, and other cross platform os things.</li> <li>We use it with some tools and it can be the bottleneck, so a speedup is very welcome.</li> </ul> <p><a href=""><strong>Pytest 3.0.4</strong></a></p> <ul> <li>Mostly a bug fix release. Check out the change log to see if you care about anything they fixed.</li> <li>Report teardown output on test failure. Already did report setup output.</li> </ul> <p><a href=""><strong>Registration is open for PyCon 2017!</strong></a></p> <ul> <li>Don't wait! Take advantage of the early bird discount </li> <li>We are having a booth!</li> </ul> <p><a href=""><strong>PyCon Tutorial proposals due on November 30</strong></a></p> <ul> <li>Tutorial proposals are due on 2017 November 30</li> <li>Talk proposals will be due on 2017 January 3</li> <li>Poster proposals will be due on 2017 January 3 We'll be hosting a Podcast open session</li> </ul> <p><a href=""><strong>Github awesome-python</strong></a></p> <ul> <li>A curated list of awesome Python frameworks, libraries, software and resources</li> <li>A sampling of the topic areas: <ul> <li>Authentication</li> <li>Data Validation</li> <li>Date and Time</li> <li>Debugging Tools</li> <li>Machine Learning</li> <li>Search</li> </ul></li> </ul> <p><a href=""><strong>Timing Tests in Python for Fun and Profit</strong></a></p> <ul> <li>Fun example of how and why you'd ever want to customize a test runner.</li> <li>Final solution is pretty cool. However, I'm not sure if it includes setup/teardown time.</li> <li>BTW, pytest has this built in, <code>pytest --durations=0</code> time every setup, teardown, and test method. <code>pytest --durations=3</code> shows the slowest 3, etc. As well as shows the time of each. Also works on unittest code.</li> </ul> <p><a href=""><strong>What's the Average Python Developer Salary in the US, and Why Is Python So Popular, Anyway?</strong></a></p> <ul> <li>2015 and 2016 have been big years for Python. According to the 2016 Developer Survey by StackOverflow, Python ranks as the world's sixth most popular programming language for the second year in a row, and is also rated the fourth most wanted technology of the year.</li> <li>Python is also one of the hottest tech skills to have, according to research by Dice, with demand slightly outstripping supply.</li> <li>Python is one of the highest-paying programming languages in the USA. In fact, at $103,492 per year or by Indeed - $116,129</li> <li>Why? <ul> <li>The world's tech giants love it. </li> <li>With the rise of data science, Python's popularity as a scientific language has soared</li> <li>It's perfect for beginners</li> <li>It might just be the only language you need</li> </ul></li> </ul> <h2>Our personal news</h2> <h3>Brian</h3> <ul> <li>T&amp;C #24: Raphael Pierzina, on cookie cutter, pytest 3.0, and contributing to both: <a href=""></a> </li> <li>T&amp;C #25: Dave Hunt, recorded. Hope to get that out this week. We talk about his work on Selenium, pytest-selenium, pytest-html, tox, and how Mozilla does some of it's testing with these tools.</li> <li>Looking for more people to interview. A few in the pipeline, including David Hussman from <a href=""></a>, but I'd love suggestions. Doesn't have to be testing. I'll be talking to David about Dude's Law, his view of modern agile practices, and other high level topics.</li> </ul> <h3>Michael</h3> <ul> <li><a href="">Python: An Amazing Second Language for .NET Developers on PyVideo</a></li> <li><a href="">Mutation Testing in Python with Cosmic Ray on PyVideo</a></li> <li>New content in my <a href="">Python for Entrepreneurs course</a></li> <li><a href="">Talk Python To Me</a> #85 is coming this week</li> </ul>
Nov 14, 2016
#1 Intro to the show and pip 9 is out!
<p>Welcome to Python Bytes. Python headlines delivered directly to your earbuds. In this first episode we cover PyData videos, safety-db project, and more! </p> <h2>News items</h2> <p><a href=""><strong>PyData DC 2016 videos are out</strong></a></p> <ul> <li>64 videos (30 hours)</li> <li>Some standout videos</li> <li>ElasticSearch and Redis How and When to Use Them</li> <li>Renee Teate | Becoming a Data Scientist Advice From My Podcast Guests (becoming a data scientist podcast)</li> <li>Visual diagnostics for more informed machine learning</li> <li>You got your engineering in my Data Science: Addressing the reproducibility crisis</li> </ul> <p><a href=""><strong>easyargs</strong></a></p> <ul> <li>A python module to make handling command line arguments easy</li> <li>There are many ways to create a command line parser in python: argparse, docopt, click. These are all great options, but require quite a lot of configuration and sometimes you just need a function to be called. Enter easyargs. Define the function that you want to be called, decorate it and let easyargs work out the command line.</li> </ul> <p><a href=""><strong>safety-db</strong> via github</a></p> <ul> <li>A curated database of insecure Python packages</li> <li><strong>What is this?</strong> This is an effort to collect all known security vulnerabilities in Python packages and make them available to consume for humans and automated tools.</li> <li><strong>What is this not?</strong> This is not a hall of shame, or a list of packages to avoid. The package maintainers show a great responsibility by documenting and fixing security issues in such a way that they can be listed here. That's extremely valuable when considering using a package in production.</li> </ul> <p><a href=""><strong>fast-style-transfer</strong></a></p> <ul> <li>Since A-Ha's take on me video, I've loved the idea of short video done in stylized art format. </li> <li>Uses Tenserflow, which I've never used before. No practical use I can think of, but would be fun.</li> </ul> <p><a href=""><strong>pip 9.0 is out!</strong></a></p> <ul> <li>New: Add a pip check command to check installed packages dependencies</li> <li>pip show is less verbose by default. </li> <li>Add --not-required option to pip list, which lists packages that are not dependencies of other packages.</li> <li>Many more, including fixes</li> <li>9.0.1 (4 days later with 5 fixes)</li> </ul> <p><a href=""><strong>What does Reddit think about the new MBP?</strong></a></p> <ul> <li>I'd never even heard of sentiment analysis before. This is cool that it's topical, interesting, and short.</li> </ul> <h2>Our personal news</h2> <h3>Brian</h3> <ul> <li>Episode 24 <a href="">Test and Code Podcast</a>: Raphael Pierzina (cookie cutter, pytest 3.0, pytest sprint, …) will be out the week of Nov 7.</li> <li>Some upcoming interviews I'm really excited about.</li> <li>Video from PNSQC 2016 conference now up. <a href=""></a></li> <li>Pietist book still in progress. When there is more info to know, I will share with this podcast, as well as people who follow me through the slack community, patreon supporters, email subscribers. </li> <li>And I'm going do my best to get $5 off for everyone who has purchased the current eBook and left me their email. </li> </ul> <h3>Michael</h3> <ul> <li>New <a href="">Talk Python To Me podcast</a> episodes <ul> <li>Are we failing to fund Python's core infrastructure?</li> <li>Parsing horrible things with Python - Erik Rose</li> <li>Python at StackOverflow - Martijn Pieters</li> </ul></li> </ul>
Nov 07, 2016