Subscribe: Planet Python
http://www.planetpython.org/rss20.xml
Added By: Feedage Forager Feedage Grade B rated
Language: English
Tags:
api  application  auth  authentication  based  cors  create  data  django  graphql  model  origin  python  rest  server  user 
Rate this Feed
Rate this feedRate this feedRate this feedRate this feedRate this feed
Rate this feed 1 starRate this feed 2 starRate this feed 3 starRate this feed 4 starRate this feed 5 star

Comments (0)

Feed Details and Statistics Feed Statistics
Preview: Planet Python

Planet Python



Planet Python - http://planetpython.org/



 



Artem Golubin: Understanding internals of Python classes

Sun, 21 Jan 2018 15:14:57 +0000

The goal of this series is to describe internals and general concepts behind the class object in Python 3.6. In this part, I will explain how Python stores and lookups attributes. I assume that you already have a basic understanding of object-oriented concepts in Python.

Let's start with a simple class:

class Vehicle:
    kind = 'car'

    def __init__(self, manufacturer, model):
        self.manufacturer = manufacturer
        self.model_name = model

    @property
    def name(self):
        return "%s %s" % 



Import Python: #159: How to speed up Python application startup time, Hunting Memory Leaks and more

Sun, 21 Jan 2018 09:34:37 +0000

Worthy Read Optimize Utilization with GoCD’s Elastic Agents GoCD is a continuous delivery tool specializing in advanced workflow modeling and dependency management. Our new AWS ECS elastic agents extension now allows you to scale up with on-demand agents based on your need. Try it now! GoCD, advert How to speed up Python application startup time? Python 3.7 has new feature to show time for importing modules. This feature is enabled with -X importtime option or PYTHONPROFILEIMPORTTIME environment variable. processing time Using textual analysis to quantify a cast of characters If you’ve ever worked on a text and wished you could get a list of characters or see how many times each character was mentioned, this is the tutorial for you. NLTK Hunting for memory leaks in asyncio applications. Sailing into the last two weeks of 2017 that I fully intended to spend experimenting with various eggnog recipes. I was alerted by our DevOps team that our asyncio app was consuming 10GB of memory. That is approximately 100 times more than it should! memory leaks, async The Industry’s Fastes[...]



Techiediaries - Django: Handling CORS in Express 4

Sun, 21 Jan 2018 00:00:00 +0000

CORS stands for Cross Origin Resource Sharing and allows modern web browsers to be able to send AJAX requests and receive HTTP responses for resource from other domains other that the domain serving the client side application. If you have ever been developing an application which is making XHR requests to a cross-domain origin and getting an error like the following in your browser console? XMLHttpRequest cannot load XXX. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin XXX is therefore not allowed access. The response had HTTP status code 500. Your web browser is simply informing you that your web server is not sending back the headers that allow CORS i.e Access-Control-Allow-Origin and Access-Control-Allow-Methods So in this tutorial you'll learn how to enable CORS in your Express 4 server to enable your front-end application to bypass the Same Origin Policy enforced by modern web browsers. This is particularly useful when you are locally developing your application, since in many cases you'll have two running development servers (front-end and back-end servers) in different ports, or if you want to enable resource sharing between different domains/hosts. How to enable CORS in Express 4 There are many ways that you can use to enable CORS in Express. If you are locally developing your application and want a quick way to CORS then you can simply use a middleware with a few lines of code: var express = require('express'); var server = express(); server.use(bodyParser.urlencoded({extended: true})) server.use(bodyParser.json()) server.use(function(req, res, next) { res.header("Access-Control-Allow-Origin", "*"); res.header("Access-Control-Allow-Headers", "Origin, X-Requested-With, Content-Type, Accept"); next(); }); server.get('/endpoint', function (req, res, next) { res.json({msg: 'This is CORS-enabled for all origins!'}) }) server.listen(3000, () => { console.log('Listenning at http://localhost:3000' ) }) The wildcard * allows resources to be accessed from any origin. That's it you can now send requests from any origin without getting the same origin policy problems. You can also use fine grained options without having to deal with HTTP header names for CORS by using the CORS module installed from npm. Using the CORS Module Head over to your terminal and install: npm install --save cors You can then use it as a middleware var express = require('express'); var server = express(); var cors = require('cors'); server.use(bodyParser.urlencoded({extended: true})) server.use(bodyParser.json()) server.get('/endpoint', function (req, res, next) { res.json({msg: 'This is CORS-enabled for all origins!'}) }) server.use(cors()); server.listen(3000, () => { console.log('Listenning at http://localhost:3000' ) }) This is equivalent to our previous example and allows resources to be accessed from any origin by adding the Access-Control-Allow-Origin: * header to all requests. Controlling Allowed Hosts When your are in production you don't want to allow CORS access for all origins but if you need to allow cross origin requests from some specified host(s) you can do add the following code: server.use(cors({ origin: 'https://techiediaries.com' })); This wil allow https://techiediaries.com to send cross origin requests to your Express server without the Same Origin Policy getting in the way. You can also enable CORS for a single Express route server.get('/endpoint', cors(), function (req, res, next) { res.json({msg: 'This has CORS-enabled for only this route: /endpoint'}) }) Allowing Dynamic/Multiple Origins If you want to allow multiple origins you need to use a function (for origin instead of a string) that dynamically set the CORS header depending on the origin making the request and a white list that you specify which contains the origin to allow. var express = require('express') var cors = require('cors') var server = express() var whitelist = ['http://techiediaries.com', 'http://othersit[...]



Python Data: Local Interpretable Model-agnostic Explanations – LIME in Python

Sat, 20 Jan 2018 19:57:22 +0000

When working with classification and/or regression techniques, its always good to have the ability to ‘explain’ what your model is doing. Using Local Interpretable Model-agnostic Explanations (LIME), you now have the ability to quickly provide visual explanations of your model(s). Its quite easy to throw numbers or content into an algorithm and get a result that looks good. We can test for accuracy and feel confident that the classifier and/or model is ‘good’…but can we describe what the model is actually doing to other users? A good data scientist spends some of their time making sure they have reasonable explanations for what the model is doing and why the results are what they are. There’s always been a focus on ‘trust’ in any type of modeling methodology but with machine learning and deep learning, many people feel like the black-box approach taken with these methods isn’t as trustworthy as other methods.  This topic was addressed in a paper titled Why Should I Trust You?”: Explaining the Predictions of Any Classifier, which proposes the concept of Local Interpretable Model-agnostic Explanations (LIME). According to the paper, LIME is ‘an algorithm that can explain the predictions of any classifier or regressor in a faithful way, by approximating it locally with an interpretable model.’ I’ve used the LIME approach a few times in recent projects and really like the idea. It breaks down the modeling / classification techniques and output into a form that can be easily described to non-technical people.  That said, LIME isn’t a replacement for doing your job as a data scientist, but it is another tool to add to your toolbox. To implement LIME in python, I use this LIME library written / released by one of the authors the above paper. I thought it might be good to provide a quick run-through of how to use this library. For this post, I’m going to mimic “Using lime for regression” notebook the authors provide, but I’m going to provide a little more explanation. The full notebook is available in my repo here. Getting started with Local Interpretable Model-agnostic Explanations (LIME) Before you get started, you’ll need to install Lime.pip install limeNext, let’s import our required libraries.from sklearn.datasets import load_boston import sklearn.ensemble import numpy as np from sklearn.model_selection import train_test_split import lime import lime.lime_tabularLet’s load the sklearn dataset called ‘boston’. This data is a dataset that contains house prices that is often used for machine learning regression examples.boston = load_boston()Before we do much else, let’s take a look at the description of the dataset to get familiar with it.  You can do this by running the following command:print boston['DESCR']The output is:Boston House Prices dataset =========================== Notes ------ Data Set Characteristics: :Number of Instances: 506 :Number of Attributes: 13 numeric/categorical predictive :Median Value (attribute 14) is usually the target :Attribute Information (in order): - CRIM per capita crime rate by town - ZN proportion of residential land zoned for lots over 25,000 sq.ft. - INDUS proportion of non-retail business acres per town - CHAS Charles River dummy variable (= 1 if tract bounds river; 0 otherwise) - NOX nitric oxides concentration (parts per 10 million) - RM average number of rooms per dwelling - AGE proportion of owner-occupied units built prior to 1940 - DIS weighted distances to five Boston employment centres - RAD index of accessibility to radial highways - TAX full-value property-tax rate per $10,000 - PTRATIO pupil-teacher ratio by town - B 1000(Bk - 0.63)^2 where Bk is the proportion of blacks [...]



Possbility and Probability: Using Python argparse and arguments with dashes

Sat, 20 Jan 2018 15:30:20 +0000

Have you ever been soooooo close to putting a puzzle together only to discover at the last minute that you are missing a piece? This happens to me all the time when I’m coding and I hit that last little … Continue reading

The post Using Python argparse and arguments with dashes appeared first on Possibility and Probability.




David MacIver: Lazy fisher-yates shuffling for precise rejection sampling

Sat, 20 Jan 2018 14:44:43 +0000

This is a trick I figured out a while ago. It came up in a problem I was working on, so I thought I’d write it up. I haven’t seen it anywhere else, but I would be very surprised if it were in any way original to me rather than a reinvention. I think it’s niche and not very useful, so it’s hard to find prior art. Attention conservation notice: My conclusion at the end is going to be that this is a neat trick that is probably not worth bothering with. You get a slight asymptotic improvement for a worse constant factor. I mention some variants and cases where it might be useful at the end, but it’s definitely more of an interesting intellectual exercise than a practical tool. Suppose you have the following problem: You want to implement the following function: def sample(random, n, f): """returns a number i in range(n) such that f(i) is True. Raises ValueError if no such i exists.""" What is the right way to do this? The obvious way to do it is this: def sample(random, n, f): choices = [i for i in range(n) if f(i)] if not choices: raise ValueError("No such i!") return random.choice(choices) We calculate all of the possible outcomes up front and then sample from those. This works but it is always \(O(n)\). We can’t help to do better than that in general – in the case where \(f(i)\) always returns False, we must call it \(n\) times to verify that and raise an error – but it is highly inefficient if, say, f always returns True. For cases where we know a priori that there is at least one i with f(i) returning True, we can use the following implementation: def sample(random, n, f): while True: i = random.randrange(0, n) if f(i): return i Here we do rejection sampling: We simulate the distribution directly by picking from the larger uniform distribution and selecting on the conditional property that \(f(i)\) is True. How well this works depends on a parameter \(A\) – the number of values in \([0, n)\) such that \(f(i)\) is True. The probability of stopping at any loop iteration is \(\frac{A}{n}\) (the probability of the result being True), so the number of loops is a geometric distribution with that parameter, and so the expected number of loop iterations is \(\frac{n}{R}\). i.e. we’ve effectively sped up our search by a factor of \(A\). So in expectation rejection sampling is always at least as good and usually better than our straightforward loop, but it has three problems: It will occasionally take more than \(n\) iterations to complete because it may try the same \(i\) more than once. It loops forever when \(A=0\) Even when \(A > 0\) its worst-case may take strictly more iterations than the filtering method. So how can we fix this? If we fix the first by arranging so that it never calls \(f\) with the same \(i\) more than once, then we also implicitly fix the other two: At some point we’ve called \(f\) with every \(i\) and we can terminate the loop and error, and if each loop iteration returns a fresh \(i\) then we can’t have more than \(n\) iterations. We can do this fairly naturally by shuffling the order in which we try the indices and then returning the first one that returns True: def sample(random, n, f): indices = list(range(n)) random.shuffle(indices) for i in indices: if f(i): return i raise ValueError("No such i!") Effectively at each iteration of the loop we are uniformly at random selecting an \(i\) among those we haven’t seen yet. So now we are making fewer calls to \(f\) – we only call it until we first find an example, as in our rejection sampling. However we’re still paying an \(O(n)\) cost for actually doing the shuffle! We can fix this partially by inlining the implementation of the shuffle as follows (this is a standard Fisher-Yates shuffle, though we’re doing it backwards: def[...]



David MacIver: A pathological example for test-case reduction

Sat, 20 Jan 2018 11:46:24 +0000

This is an example I cut from a paper I am currently writing about test-case reduction because I needed to save space and nobody except me actually cares about the algorithmic complexity of test-case reduction. Suppose you have the following test case: from hypothesis import given, strategies as st K = 64 INTS = st.integers(0, 2 ** K - 1) @given(INTS, INTS) def test_are_not_too_far_apart(m, n): assert abs(m - n) > 1 Then if this test ever fails, with initial values \(m\) and \(n\), if reduction can replace an int with its predecessor but can’t change two ints at once, the reduction will take at least \(\frac{m + n}{2}\) test executions: The fastest possible path you can take is to at each step reduce the larger of \(m\) and \(n\) by two, so each step only reduces \(m + n\) by \(2\), and the whole iteration takes \(\frac{m + n}{2}\) steps. (Side note: You can get lucky with Hypothesis and trigger some special cases that make this test faster. if you happen to have \(m = n\) then it can reduce the two together, but if you start with \(m = n \pm 1\) then it will currently never trigger because it will not ever have duplicates at the entry to that step. Hypothesis will also actually find this bug immediately because it will try it with both examples set to zero. Trivial modifications to the test can be made to avoid these problems, but I’m going to ignore them here). The interesting thing about this from a Hypothesis point of view is that \(m + n\) is potentially exponential in \(k\), and the data size is linear in \(k\), so Hypothesis’s test case reduction is of exponential complexity (which doesn’t really cause a performance problem in practice because the number of successful reductions gets capped, but does cause an example quality problem because you then don’t run the full reduction). But this isn’t specifically a Hypothesis problem – I’m morally certain every current property-based testing library’s test case reduction is exponential in this case (except for ones that haven’t implemented reduction at all), possibly with one or two patches to avoid trivial special cases like always trying zero first. Another way to get around this is to almost never trigger this test case with large values! Typically property-based testing libraries will usually only generate an example like this with very small values. But it’s easy for that not to be the case – mutational property-based testing libraries like Hypothesis or crowbar can in theory fairly easily find this example for large \(m\) and \(n\) (Hypothesis currently doesn’t. I haven’t tried with Crowbar). Another way you could easily trigger it is with distributions that special case large values. One thing I want to emphasise is that regardless of the specific nature of the example and our workarounds for it, this sort of problem is inherent. It’s easy to make patches that avoid this particular example (especially in Hypothesis which has no problem making simultaneous changes to \(m\) and \(n\)). But if you fix this one, I can just construct another that is tailored to break whatever heuristic it was that you came up with. Test-case reduction is a local search method in an exponentially large  space, and this sort of problem is just what happens when you block off all of the large changes your local search method tries to make but still allow some of the small ones. You can basically keep dangling the carrot in front of the test-case reducer going “Maybe after this reduction you’ll be done”, and you can keep doing that indefinitely because of the size of the space. Pathological examples like this are not weird special cases, if anything the weird thing is that most examples are not pathological like this. My suspicion is that we don’t see this problem cropping up much in practice f[...]






Amjith Ramanujam: FuzzyFinder - in 10 lines of Python

Sat, 20 Jan 2018 03:09:19 +0000

Introduction:FuzzyFinder is a popular feature available in decent editors to open files. The idea is to start typing partial strings from the full path and the list of suggestions will be narrowed down to match the desired file. Examples: Vim (Ctrl-P) Sublime Text (Cmd-P) This is an extremely useful feature and it's quite easy to implement.Problem Statement:We have a collection of strings (filenames). We're trying to filter down that collection based on user input. The user input can be partial strings from the filename. Let's walk this through with an example. Here is a collection of filenames:When the user types 'djm' we are supposed to match 'django_migrations.py' and 'django_admin_log.py'. The simplest route to achieve this is to use regular expressions. Solutions:Naive Regex Matching:Convert 'djm' into 'd.*j.*m' and try to match this regex against every item in the list. Items that match are the possible candidates.This got us the desired results for input 'djm'. But the suggestions are not ranked in any particular order.In fact, for the second example with user input 'mig' the best possible suggestion 'migrations.py' was listed as the last item in the result.Ranking based on match position:We can rank the results based on the position of the first occurrence of the matching character. For user input 'mig' the position of the matching characters are as follows:Here's the code:We made the list of suggestions to be tuples where the first item is the position of the match and second item is the matching filename. When this list is sorted, python will sort them based on the first item in tuple and use the second item as a tie breaker. On line 14 we use a list comprehension to iterate over the sorted list of tuples and extract just the second item which is the file name we're interested in.This got us close to the end result, but as shown in the example, it's not perfect. We see 'main_generator.py' as the first suggestion, but the user wanted 'migration.py'.Ranking based on compact match:When a user starts typing a partial string they will continue to type consecutive letters in a effort to find the exact match. When someone types 'mig' they are looking for 'migrations.py' or 'django_migrations.py' not 'main_generator.py'. The key here is to find the most compact match for the user input.Once again this is trivial to do in python. When we match a string against a regular expression, the matched string is stored in the match.group(). For example, if the input is 'mig', the matching group from the 'collection' defined earlier is as follows:We can use the length of the captured group as our primary rank and use the starting position as our secondary rank. To do that we add the len(match.group()) as the first item in the tuple, match.start() as the second item in the tuple and the filename itself as the third item in the tuple. Python will sort this list based on first item in the tuple (primary rank), second item as tie-breaker (secondary rank) and the third item as the fall back tie-breaker. This produces the desired behavior for our input. We're not quite done yet.Non-Greedy MatchingThere is one more subtle corner case that was caught by Daniel Rocco. Consider these two items in the collection ['api_user', 'user_group']. When you enter the word 'user' the ideal suggestion should be ['user_group', 'api_user']. But the actual result is:Looking at this output, you'll notice that api_user appears before user_group. Digging in a little, it turns out the search user expands to u.*s.*e.*r; notice that user_group has two rs, so the pattern matches user_gr instead of the expected user. The longer match length forces the ranking of this match down, which again seems counterintuitive. Th[...]



Sandipan Dey: Hand-Gesture Classification using Deep Convolution and Residual Neural Network with Tensorflow / Keras in Python

Sat, 20 Jan 2018 01:22:56 +0000

In this article, first an application of convolution net to classify a set of hand-sign images is going to be discussed.  Later the accuracy of this classifier will be improved using a deep res-net. These problems appeared as assignments in the Coursera course Convolution Neural Networks (a part of deep-learning specialization) by the Stanford Prof. Andrew Ng. (deeplearning.ai). The problem descriptions are taken straightaway … Continue reading Hand-Gesture Classification using Deep Convolution and Residual Neural Network with Tensorflow / Keras in Python



Techiediaries - Django: An Introduction to REST with PHP by Building a Simple Real World API

Sat, 20 Jan 2018 00:00:00 +0000

In this tutorial we are going to learn how to build a simple real world REST API with plain PHP. This API will be the basis of the next tutorials for adding JWT-based authentication and building your front-ends with modern JavaScript/TypeScript frameworks and libraries such as Angular, React and Vue etc. Throughout the tutorial we'll create a simple API (but in the same time it's a real-world API. In fact you can use it to build a small stock tracking app) with the most straightforward and simplest architecture (and file structure) i.e we are not going to cover advanced concepts such as MVC, routing or template languages (we will use PHP itself as the template language. I know it's a bad practice but this is how you do things when you first get started using PHP also if you are looking for these concepts you better use a PHP framework, most of them are built around these advanced concepts) so this tutorial can be as beginners-friendly as possible. What is an API? API stands for Application Interface Programming. It's an interface that allows applications to communicate with each other. In case of the web it refers to the interface (a set of URLs that allows you to exchange data with a web application via a set of operations commonly known as CRUD--Create, Read, Update and Delete operations by sending HTTP requests such as POST, GET, PUT and DELETE etc. What is REST? REST stands for REpresentational State Transfer". It's a set of rules that define how to exchange resources in a distributed system such as stateleness i.e the server doesn't keep any information about the previous requests which means the current request should include every information the server needs for fulfilling the desired operation. Data is usually exchanged in JSON (JavaScript Object Notation) format. So REST API refers to the interface that allows mobile devices and web browsers (or also other web servers) to create, read, update and delete resources in the server respecting the REST rules (such as being stateless). Using REST you can build one back-end and then build different client apps or front-ends for web browsers and mobile devices (iOS and Android etc.) because the back-end is decoupled from the front-end--the communication between the client and the server apps takes place via the REST interface. You can offer your users another app or you can build more apps to support the other mobile platforms without touching the back-end code. Database Design In order to build a web API, you need a way to store data, behind the scene, in your server's database. For this tutorial we'll use MySQL RDMS (Relational Database Management System) which is the most used database system in the PHP world. The first step is to design our database so we'll use the Entity-Relationship Diagram An entity relationship diagram, or also an entity-relationship model, is a graphical representation of entities and how they relate to each other. They are used to model relational databases. In ER diagrams you use entities (boxes) to represent real world concepts or objects and relationships (arrows) to represent a relation between two entities. There are three types of relationships: One-to-One, One-to-Many and Many-to-Many. Here is a screenshot of an example ER model for our database We have four entities that are related with each other: A product has a family, belongs to a location and can have many related transactions. After creating an ER model you can easily write SQL CREATE statements to create the SQL tables in the MySQL database. You can simply map each entity to a SQL table and relationships to foreign keys. Any decent ER diagramming tool will include an export button that can help you generate the SQL script from your ER model without having to write it m[...]



Techiediaries - Django: Building an Ionic 3/Angular 4|5 Application with a GraphQL API

Sat, 20 Jan 2018 00:00:00 +0000

Let's see how to build a CRUD Ionic 3 mobile application (or if you prefer an Angular 4+ web application) using the modern GraphQL-based API (instead of REST-based API). Since we need a backend to serve the API, we'll cover two options, we'll first see how to use the GraphQL Launchpad to easilly create an Apollo server in 60 lines of code then we'll see how you can build your own self hosted backend using Python and Django Introduction to GraphQL GraphQL is a modern standard for creating web Application Programming Interfaces, commonly known as web APIs. For the last decade REST become the standard way for building APIs but thanks to Facebook there is now a more powerful alternative that have many advantages over REST. GraphQL is a query language for APIs and a runtime for fulfilling those >queries with your existing data. GraphQL provides a complete and >understandable description of the data in your API, gives clients the power >to ask for exactly what they need and nothing more, makes it easier to >evolve APIs over time, and enables powerful developer tools. --- graphql.org So let's break this: GraphQL is a standard and runtime for building APIs not a programming language or a developer tool GraphQL gives you more control over your data i.e you can specify exactly what data attributes you want returned with the same query. You can use one query to fetch related-data Unlike Rest APIs you are not dependent of the server implementation of your endpoints less round-trips to the server for getting all data you need GraphQL is created and used internally by Facebook then open sourced in 2015. Many big industry players are using GraphQL for implemeting their API layer such as: Facebook: the creator of GraphQL Github Shopify Product Hunt Pinterest Twitter yelp etc. You can find more companies that are using GraphQL from this link. Queries and Mutations For working with GraphQL, you need to be familair with two important concepts which are Queries and Mutations. Queries are used to query and retrieve data from the GraphQL server. For example, suppose you want to get the list of products from a GraphQL backend. Here is an example of a query you would send: query { products{ id, reference, quantity } } A Query is a JSON object which has a root field and a payload(a set of fields). Using a query, you can specify the name and the exact fields of the object to retrieve. You can also pass parameters to a query, for example for getting an object with its id. Here an example of a query you would send: query { product(id: 1) { id, reference } } You can also nest a query inside another query for getting related objects data. For example to get the products of the queryied families you would send something like: query { famillies { id, reference, products{ id, reference } } } A mutation is a write that generally returns the newly modified data object (as opposed to a query, which is meant to be read-only) Using GraphQL: Servers You can either build a backend which exposes a GraphQL-based API using your preferred language such as JavaScript/Node, PHP, Python or Ruby etc. Or you can also GraphQL based hosted services or headless content management systems such as: GraphCMS - GraphQL based Headless Content Management System. Graphcool - Your own GraphQL backend in under 5 minutes. Works with every * GraphQL client such as Relay and Apollo. Reindex - Instant GraphQL Backend for Your React Apps. Scaphold - GraphQL as a service that includes API integrations such as Stripe and Mailgun. Tipe - Next Generation API-first CMS with a GraphQL or REST API. Stop letting your CMS decide how you[...]



Techiediaries - Django: 3+ Ways to Add Authentication to Ionic 3 (Angular 4|5) Applications

Sat, 20 Jan 2018 00:00:00 +0000

Let's look at the available options for adding authentication (login and registration) into your mobile application built using Ionic 3 and Angular 4|5 such as SaaS (Software As a Service) providers like Firebase, Auth0 and Okta, free third party (Single Sign On) services like Facebook, GitHub and Google, self hosted servers like Parse or building your own auth back-end with PHP, Python, Ruby or Node etc. More often than not, when building your Ionic 3 mobile application (for Android or iOS), NativeScript mobile app or your Angular 4|5 web application, you would want to authenticate users with a remote HTTP server before authorizing them to access some protected resource(s) or restful API endpoint(s). You would say that's authorization not authentication? You are correct! Authentication i.e verifying the identity of a user is the simplest form of authorization (You can of course build a more advanced authorization system but that's not required in most cases except for multi-tenant apps where there are many users with different roles for the same account). I recently intended to build an Ionic app with authentication so I looked for the available choices to build an authentication system with different features such as login, signup, user verification and password recovery via email etc. And found that there are many viable options from building your own hosted solution with a back-end technology, if you've got the required skills in some server side language such as PHP or Python (Django or Flask etc.) to hosted solutions (such as Firebase or Auth0) that allow you to build a back-end for your mobile/web applications with authentication, data storage and many extra features without the prior knowledge of a server side language, without reinventing the wheel and without hiring a back-end developer. First of all, this article is not intended to show you how to create an Ionic 3 project since we have previously covered this in many tutorials. With Ionic 3 and Angular you can literally build a fully fledged and complete mobile application for popular platforms such as Android, iOS and the Windows Universal Platform around these hosted services (we'll see them next) or around your own crafted back-end (but it's not that easy if you are not a skilled server side developer) In this article, we'll look briefly at different ways to build an authentication system in Ionic 3 and Angular 4 without in-depth details on how to use each option but I will add links for more detailed tutorials on specific technologies if they are available or update the article once I have time to write more tutorials. Also please feel free to ask for a specific tutorial or for more information using the comments area below or via twitter(@techiediaries). Adding User Authentication with SaaS/PaaS Services Wikipedia defines SaaS as: Software as a service is a software licensing and delivery model in which software is licensed on a subscription basis and is centrally hosted. It is sometimes referred to as "on-demand software",and was formerly referred to as "software plus services" by Microsoft. SaaS is typically accessed by users using a thin client via a web browser. SaaS has become a common delivery model for many business applications, including office software, messaging software, payroll processing software, DBMS software, management software etc. So simply put, a SaaS is a software delivery model i.e a way to deliver software, to users, without downloading it from the Internet or copying it from a USB/CD medium and installing it in the local machine. Also from Wikipedia, here is the definition of PaaS Platform as a Service (PaaS) or application platform as a Service (aPaaS) is a category of c[...]



Techiediaries - Django: Adding JWT Authentication to Python and Django REST Framework Using Auth0

Sat, 20 Jan 2018 00:00:00 +0000

In this tutorial we'll learn how to add JWT authentication to an API built with Django REST framework. Basically we'll use the djangorestframework-jwt package for adding JWT authentication as you would normally do except that we'll change JWT_AUTH to use Auth0. This tutorial assumes you already have a development machine with Python 3 and pip installed and will cover the following points: We'll see how to create a virtual environment, install Django and the other dependencies (Django REST framework and djangorestframework-jwt) We'll see how to create an Auth0 API We'll see how to integrate Auth0 JWT authentication with Django We'll briefly talk about using Auth0 Rules for detecting signup We'll see how to add some Django views for testing JWT We'll see how to use Postman for testing JWT authentication with Auth0 Creating the Django Project So head over to your terminal then create a new virtual environment and activate it using the venv module in your current working directory: python3 -m venv ./myenv source myenv/bin/activate Next install Django using pip: pip install django Now you'll need to create a new Django project using: django-admin startproject auth0-django-example Next create a new application in your project cd auth0-django-example python manage.py startapp customers Add customers to the installed apps in your project' settings.py file: INSTALLED_APPS = [ 'django.contrib.admin', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.messages', 'django.contrib.staticfiles', 'customers' ] Next migrate your database then start the development server python manage.py migrate python manage.py runserver You can visit your app at http://localhost:8000 Create an Auth0 API Head over to your Auth0 dashboard then create an API Go to the API section then click on the CREATE API button which will show a form where you need to enter your API details Integrating Auth0 with Django Now head back to your terminal then install Django REST framework and djangorestframework-jwt package for handling JWT authentication using pip pip install djangorestframework pip install djangorestframework-jwt pip install cryptography pip install python-jose Add rest_framework and rest_framework_jwt to the installed apps in settings.py: INSTALLED_APPS = [ 'rest_framework', 'rest_framework_jwt' ] Next you'll need to setup djangorestframework-jwt to use Auth0 central server for JWT authentication by follwing a few steps. First add JSONWebTokenAuthentication to DEFAULT_AUTHENTICATION_CLASSES: REST_FRAMEWORK = { 'DEFAULT_PERMISSION_CLASSES': ( 'rest_framework.permissions.IsAuthenticated', ), 'DEFAULT_AUTHENTICATION_CLASSES': ( 'rest_framework_jwt.authentication.JSONWebTokenAuthentication', ), } Secondly import the follwing libs in your settings.py file: import json from six.moves.urllib import request from cryptography.x509 import load_pem_x509_certificate from cryptography.hazmat.backends import default_backend Finally add this code to settings.py: AUTH0_DOMAIN = '' API_IDENTIFIER = '' PUBLIC_KEY = None JWT_ISSUER = None if AUTH0_DOMAIN: jsonurl = request.urlopen('https://' + AUTH0_DOMAIN + '/.well-known/jwks.json') jwks = json.loads(jsonurl.read().decode('utf-8')) cert = '-----BEGIN CERTIFICATE-----\n' + jwks['keys'][0]['x5c'][0] + '\n-----END CERTIFICATE-----' certificate = load_pem_x509_certificate(cert.encode('utf-8'), default_backend()) PUBLIC_KEY = certificate.public_key() JWT_ISSUER = 'https://' + AUTH0_DOMAIN + '/' def jwt_get_username_from_payload_handler(payload): return 'som[...]



Techiediaries - Django: QuickTip: Django and AngularJS Conflicting Interpolation Symbols

Sat, 20 Jan 2018 00:00:00 +0000

When using the Django framework with the AngularJS MVC framework for building modern single page applications or SPAs, one of the issues you will encouter is related to both frameworks using the same symbols for template tags i.e { { and } }. So in this quick tip post we'll see how to change the interpolation symbols in AngularJS to avoid these conflicts. Luckliy for us, AngularJS provides the $interpolateProvider provider which allows developers to customize the interpolation symbols which default to { { and } }. Used for configuring the interpolation markup. Defaults to . This feature is sometimes used to mix different markup languages, e.g. to wrap an AngularJS template within a Python Jinja template (or any other template language). Mixing templating languages is very dangerous. The embedding template language will not safely escape AngularJS expressions, so any user-controlled values in the template will cause Cross Site Scripting (XSS) security bugs! -- https://docs.angularjs.org/api/ng/provider/$interpolateProvider Simple AngularJS Example Let's see a simple example: Go ahead and create a base template ng-base.html file in your templates folder then add the following content { % load staticfiles % } Integrate Angular 1.6.6 with Django
{ % block content % }{ % endblock content % }
{$ ctrl.mymodel $}
Next create js/app in your project's static folder and add the following code to create a new AngularJS app and inject $interpolateProvider in the config function. 'use strict'; var app = angular.module('demoApp', []); app.config(function($locationProvider,$interpolateProvider){ $locationProvider.html5Mode({ enabled:true }); $interpolateProvider.startSymbol('{$'); $interpolateProvider.endSymbol('$}'); }); app.controller('TestController', function() { this.mymodel = "I'm using the custom symbols"; }); So we have inject the interpolation provider $interpolateProvider then used two methods $interpolateProvider.startSymbol('{$'); and $interpolateProvider.endSymbol('$}'); to change the default sysmbols to custom ones. Now you can use { { and } } for Django templates and {$ and $} for AngularJS templates.[...]






PyCharm: PyCharm 2018.1 EAP Starts now!

Fri, 19 Jan 2018 16:35:53 +0000

We’re excited to start the Early Access Program (EAP) for PyCharm 2018.1! The next version of PyCharm will be released in the Spring. However, as we think user feedback is very important to allow us to make a better product, you can already download a preview version now.

As PyCharm 2018.1 is in development during the EAP phase, not all new features are already available, and more features will be added in the coming weeks. Also, as this is early pre-release software, it is not as stable as our release versions. Furthermore, we may decide to change and/or drop certain features as the EAP progresses.

All EAP versions will ship with a built-in EAP license, which means that these versions are free to use for 30 days after the day that they are built. As EAPs are released weekly, you’ll be able to use PyCharm Professional Edition EAP for free for the duration of the EAP program, as long as you upgrade at least once every 30 days.

New in 2018.1 EAP 1

Scientific Project Type

(image)

You can now create a new scientific project straight from the new project screen. Scientific projects are created by default with a Conda environment and will scaffold the directory for your data.

Additionally, SciView now works with remote interpreters.

Improved HiDPI Support

We now support configurations running Windows 8.1 or higher, with multiple displays that have different scale factors or a display with a fractional scale factor (anything other than 100% or 200%).

Open Terminal from the Project Tool Window

You can now right-click a folder in the Project tool window and start a terminal in that folder:

(image)

Better Code Completion for Python

We’ve improved stubs for the Python standard library to improve code completion for these libraries.

You can read about further improvements in the release notes.

Interested?

Download PyCharm 2018.1 EAP 1 from our website. Alternatively, you can use the JetBrains Toolbox App to stay up to date throughout the entire EAP.

If you’re on Ubuntu 16.04 or later, you can use snap to get PyCharm EAP, and stay up to date. You can find the installation instructions on our website.

(image)



Django Weekly: #66: DjangoCon Japan, 2.0 Tutorial for Beginners, Kubernetes, Conditional aggregation and more

Fri, 19 Jan 2018 16:09:35 +0000

Worthy ReadOptimize Utilization with GoCD’s Elastic AgentsGoCD is a continuous delivery tool specializing in advanced workflow modeling and dependency management. Our new AWS ECS elastic agents extension now allows you to scale up with on-demand agents based on your need. Try it now! advertDjango 2.0 Tutorial for Beginners - Free BookLearn web development with Python & Django. bookDjangoCon JP 2018 - First DjangoCon in JapanDjangoCon JP 2018 starts May 19th! Submit your talks until March 12th. Hosted at Cybozu in Tokyo. djangoconAidas Bendoraitis (@DjangoTricks)Twitter handle to follow for django tips and tricks. twitterThe Industry’s Fastest eSignature API IntegrationEmbed docs directly on your website with a few lines of code. Test the API for free. advertDjango — render HTML to PDFLets look through rendering our templates to a PDF. Not just some fixed values lets build a report pulling through some data from a database, and then send said PDF as an attachment to a user. pdfHackAssistant: a hackathon registration platform - Made in DjangoManaging hackers applications is probably the worst nightmare hackathon organizers have. For this reason, after struggling with applications during HackUPC Fall 2016, I decided to use my knowledge to build a tool to make it easier: HackAssistant. Now it’s deployed for HackCU at my.hackcu.org. user submissionKubernetes, Local to Production with DjangoThis section focuses on implementing the kubernetes hello-minikube tutorial adapted to a conventional Django application. The codebase for this tutorial can be cloned from my github repo and we will be working with the getting-started branch. The kubernetes version for this tutorial is assumed to be 1.9. kubernetesHere’s a dead simple React-Django setup for your next projectHere’s a simple setup for creating a React-Django project. With just a few tweaks, you can swap out React for Angular or Vue and everything would work fine. boilerplateResults of the DSF Board electionThe DSF membership elected a new board last month. The six elected directors of the DSF for 2018. DSFConditional aggregation in Django 2.0SQL, Django 2.0Django Hardcopy - Render PDFs and PNGs in Django with headless ChromeChrome introduced headless mode in v59 opening the possibility of using Chrome as a fast and elegant way of generating PDF data or PNG screenshots programatically via HTML. django-hardcopy is an alternative to other projects which leverage wkhtmltopdf, a great tool but one that lacks the portability, ease of installation, the performance, and reliability of Chrome. chrome, pdf, png Projectsdjango-progressiveimagefield - 13 Stars, 2 ForkA Django ImageField that offers progressive image loading during HTML rendering.django-db-introspection - 4 Stars, 1 ForkDjango database introspection tool, through the data table name can dynamically create a Django models ready-to-use objects. Just need a table name, you can immediately get a usable Django model, and immediately query, update and other operations, out of the box. The current version does not support foreign key relationships.20180116-FullStack-Day - 3 Stars, 0 Forkrepo for the Python / HTML / CSS / JavaScript / Django Full Stack course starting 1/16/2018[...]



Marcos Dione: callable-choices-for-django-rest-framework

Fri, 19 Jan 2018 12:18:03 +0000

At work I'm writing an API using Django/DRF. Suddenly I had to write an application (just a few pages for calling a few endpoints), so I (ab)used DRF's Serializers to build them. One of the problems I faced while doing this was that DRF's ChoiceField accepts only a sequence with the values for the dropdown, unlike Django's, who also accepts callables. This means that once you gave it a set of values, it never ever changes, at least until you restart the application. Unless, of course, you cheat. Or hack. Aren't those synonyms? class UpdatedSequence: def __init__(self, update_func): self.update_func = update_func self.restart = True self.data = None self.index = 0 def __iter__(self): # we&aposre our own iterator return self def __next__(self): # if we&aposre iterating from the beginning, call the function # and cache the result if self.restart: self.data = self.update_func() self.index = 0 try: datum = self.data[self.index] except IndexError: # we reached the limit, start all over self.restart = True raise StopIteration else: self.index += 1 self.restart = False return datum This simple class tracks when you start iterating over it and calls the function you pass to obtain the data. Then it iterates over the result. When you reach the end, it marks it to start all over, so the next time you iterate over it, it will call the function again. The function you pass can be the all() method of a QuerySet or anything else that goes fetch data and returns an iterable. In my case in particular, I also added a TimedCache so I don't read twice the db to fill two dropdown with the same info in the same form: class TimedCache: &apos&apos&aposA function wrapper that caches the result for a while.&apos&apos&apos def __init__(self, f, timeout): self.f = f self.timeout = timeout self.last_executed = None self.cache = None self.__name__ = f.__name__ + &apos (Cached %ds)&apos % timeout def __call__(self): now = time.monotonic() if self.cache is None or (now - self.last_executed) > self.timeout: self.cache = self.f() self.last_executed = now return self.cache I hope this helps someone. python django drf[...]



Talk Python to Me: #147 Quart: Flask, but 3x faster

Fri, 19 Jan 2018 08:00:00 +0000

There has been a bunch of new Python web frameworks coming out in the past few years. Generally, these have been focused solely on Python 3 and have tried to leverage Python's new async and await features.



PyCharm: PyCharm 2017.3.3 Out Now

Thu, 18 Jan 2018 17:01:45 +0000

Today we’ve published PyCharm 2017.3.3 which has a number of important fixes specific to PyCharm as well as numerous platform fixes. Get this new version now from our website!

Some of the fixes in this build:

  • Fixes for false positives in SQLAlchemy support
  • A fix for Python Console not showing a prompt after calling input()
  • A fix for freezes on closing the IDE while connecting to an SSH host
  • Various fixes for javascript and database support
  • And more, see the release notes for details

Get the new version now from our website. You can also update PyCharm from within PyCharm (Help | Check for Updates), or by using the JetBrains Toolbox App. If you’re on Ubuntu 16.04 or later, and using snap to install PyCharm, your PyCharm should automatically be updated. If you haven’t installed PyCharm using snap yet, you can do so with the following command:

sudo snap install [pycharm-professional|pycharm-community] --classic

(image)



Simple is Better Than Complex: How to Implement Multiple User Types with Django

Thu, 18 Jan 2018 12:00:00 +0000

This is a very common problem many developers face in the early stages of the development of a new project, and it’s also a question I get asked a lot. So, I thought about sharing my experience with previous Django projects on how to handle multiple user types. I’ve tried many different strategies. In this tutorial I will share my thoughts on this particular topic and share with you the strategies that worked best for me, depending on the requirements of the project. Many of the design decisions depends on the requirements and business model of the application you are developing. I will try to cover as many different scenarios as possible. Read carefully and pick the best option for you. If you learn better with examples or you are in a hurry right now, jump to the practical example. Otherwise, keep reading. Rules of Thumb What you are going to read next is not written in a stone. It’s just some general recommendations that fits most cases. If you have a good reason, or if not following those recommendations will result in a better application design, go ahead and break the “rules”! 1. No matter what strategy you pick, or what is your business model, always use one, and only one Django model to handle the authentication. You can still have multiple user types, but generally speaking it’s a bad idea to store authentication information across multiple models/tables. Treat this model as an account rather than a user. Meaning, all users need an account to log in. The business logic will be implemented in a different way, so no need to have username/password spread across multiple tables. After all, all accounts should share many common resources such as login, logout, password resets, password change. 2. Never user the built-in Django User model directly, even if the built-in Django User implementation fulfill all the requirements of your application. At least extend the AbstractUser model and switch the AUTH_USER_MODEL on your settings. Requirements always change. You may need to customize the User model in the future, and switching the AUTH_USER_MODEL after your application is in production will be very painful. Mainly because you will need to update all the foreign keys to the User model. It can be done, but this simple measure (which, honestly, is effortless in the beginning of the project) can save you from headaches in the future. This will give you freedom to add custom methods to the User model without having to rely on a OneToOne model or having to implement an Abstract model. Also, the built-in Django model have some old design decisions (which are kept that way because of backwards compatibility) that are not compatible with many application requirements such as a nullable email field, the email field is not unique, the username field is case sensitive, which means you can have a user with username ana and another one with Ana, first name and last name fields which are not “internationally friendly” (some applications are better off having “full name” and “screen name” for example). Strategies How you are going to implement multiple user types depends on the requirements of your application. Below, a few questions you have to ask yourself: How much information specific to each type of user you need to maintain? Can the users have more than one role in the application? E.g. can a User be a Student and Teacher at the same time? How many different types of users the application will need to manage? A v[...]



Michael Droettboom: Jupyter notebooks and version control

Thu, 18 Jan 2018 05:00:00 +0000

Summary: Presents an experimental alternative file format for Jupyter notebooks that plays nicer with version control. The problem There's no doubt that Jupyter notebooks are taking over the world for data science experimentation. When notebooks are relied on for ongoing decision-making within an organization, it's inevitable that, like all software, they will require bugfixes or updates. Without proper version control of these changes, its difficult to know what changes were made, and, more importantly, to reason about what effect those changes may have had on the results. While you can simply put Jupyter notebooks into a version control system (VCS), such as git, the design of the notebook file format makes certain important operations, like calculating the difference between two revisions, less friendly than they could be. For example, the Jupyter notebook file format contains binary blobs for image output. The diffs between these sections are large and noisy, and ultimately unhelpful to the software developer reviewing a pull request. Secondly, since the file format is based on JSON, multi-line strings (such as all source code) are full of boilerplate: Each line is in its own set of double-quotes, with explicit newlines (\n). For example, this Python: items = [] for i, item in enumerate(database.all_docs(params={'include_docs' : True})): if i > 1: break items.append(item) print items turns into the following JSON: "source": [ "items = []\n", "for i, item in enumerate(database.all_docs(params={'include_docs' : True})):\n", " if i > 1: break\n", " items.append(item)\n", "print items" ] All of this makes it more difficult to see meaningful changes through all the noise. Some solutions I'm by no means the first person to notice these issues, and others have tackled this problem from different directions. In the blog post Making Git and Jupyter play nice, Tim Staley suggests filtering the notebook files to remove the output cells and less important content (such as metadata or execution_count). While this goes a long way to removing a lot of the "noise" in diffs, the content is still JSON. ipymd perhaps comes the closest to solving this problem, in my opinion, by converting notebooks to markdown, with code inserted as standard markdown code blocks. For example: Here is some Python code: ```python >>> print("Hello world!") Hello world! ``` It currently has a major shortcoming, in that it doesn't handle non-textual output cells. Jupyter output cells are more complex than they might appear at first glance, as each can include multiple representations of the same thing, allowing the front-end to ultimately choose the best based on context. I think there are probably some clever ways that could be resolved, but any solution is likely to be lossy relative to what standard Jupyter notebooks can do, or break strict compatibility with markdown. There is also nbdime: a tool designed specifically for diffing and merging Jupyter notebooks. The advantage of nbdime is that, since it was purpose-built for Jupyter notebooks, the user interface can take advantage of notebook-specific features, such as image diffing. Unfortunately, it is hard to integrate it into existing code review workflows, like Github pull requests. Here at Mozilla, for example, many of those doing data science in Jupyter notebooks are Firefox softwa[...]



François Dion: Readings in Visualization

Wed, 17 Jan 2018 20:41:09 +0000

"Ex-Libris" part V: Visualization


Part 5 of my "ex-libris" of a Data Scientist is now available. This one is about visualization.

Starting from a historical perspective, particularly of statistical visualization, and covering a few classic must have books, the article then goes on to cover graphic design, cartography, information architecture and design and concludes with many recent books on information visualization (specific Python and R books to create these were listed in part IV of this series). In all, about 66 books on the subject.

Just follow the link to the LinkedIn post to go directly to it:



(image)
From Jacques Bertin’s Semiology of Graphics

"Le plus court croquis m'en dit plus long qu'un long rapport", Napoleon Ier

See also

Part I was on "data and databases": "ex-libris" of a Data Scientist - Part i
Part II, was on "models": "ex-libris" of a Data Scientist - Part II

Part III, was on "technology": "ex-libris" of a Data Scientist - Part III
Part IV, was on "code": "ex-libris" of a Data Scientist - Part IV
Part VI will be on communication. Bonus after that will be on management / leadership.
Francois Dion
@f_dion

P.S.
Je vais aussi avoir une liste de publications en francais
En el futuro cercano voy a hacer una lista en espanol tambien



Stack Abuse: Levenshtein Distance and Text Similarity in Python

Wed, 17 Jan 2018 18:19:36 +0000

Introduction Writing text is a creative process that is based on thoughts and ideas which come to our mind. The way that the text is written reflects our personality and is also very much influenced by the mood we are in, the way we organize our thoughts, the topic itself and by the people we are addressing it to - our readers. In the past it did happen that two or more authors had the same idea, wrote it down separately, published it under their name and created something that was very similar. Prior to electronic publications their ideas took a while to circulate and therefore led to conflicts about the real inventor and who is the one to be honored for it. Today, every article becomes immediately available online in a digital format. Online articles are indexed correctly and linked to other documents, which makes it easy to find them quickly. On the one hand this way of working simplifies the exchange of ideas as well as the research about a topic but on the other hand the accessibility opens doors to just copy and paste others work without permission or even referencing them, called plagiarism. At this point methods come into play that deal with the similarity of two different texts. The main idea behind this is to be able to answer the questions if two texts (or datasets in general) are entirely or at least partly similar, if they are related to each other in terms of the same topic and how many edits have to be done to come from one text to the other one. As an example, this technology is used by information retrieval systems, search engines, automatic indexing systems, text summarizers, categorization systems, plagiarism checkers, speech recognition, rating systems, DNA analysis, and profiling algorithms (IR/AI programs to automatically link data between people and what they do). Search and Comparison Methods All of us are familiar with searching a text for a specified word or character sequence (pattern). The goal is to either find the exact occurrence (match) or to find an in-exact match using characters with a special meaning, for example by regular expressions or by fuzzy logic. Mostly, it is a sequence of characters that is similar to another one. Furthermore, the similarity can be measured by the way words sound - do they sound similar, but written in a different way? Invented in the U.S. to find relatives based on the different spelling of their surname, among others, the Soundex algorithm is still one of the most popular and widespread ones today. Last but not least, how many changes (edits) are necessary to get from one word to the other one? The less edits to be done the higher is the similarity level. This category of comparison contains the Levenshtein distance that we will focus on in more detail below. Table 1 covers a selection of ways to search and compare text data. The right column of the table contains a selection of the corresponding Python modules to achieve these tasks. Category Method or Algorithm Python packages exact search Boyer-Moore string search, Rabin-Karp string search, Knuth-Morris-Pratt (KMP), Regular Expressions string, re, Advas in-exact search bigram search, trigram search, fuzzy logic Fuzzy phonetic algorithms So[...]