Subscribe: O'Reilly Radar - Insight, analysis, and research about emerging technologies
http://radar.oreilly.com/atom.xml
Added By: Feedage Forager Feedage Grade B rated
Language: English
Tags:
continue reading  continue  data  docker  learning  links  machine learning  machine  new  reading  short links  short  time 
Rate this Feed
Rate this feedRate this feedRate this feedRate this feedRate this feed
Rate this feed 1 starRate this feed 2 starRate this feed 3 starRate this feed 4 starRate this feed 5 star

Comments (0)

Feed Details and Statistics Feed Statistics
Preview: O'Reilly Radar - Insight, analysis, and research about emerging technologies

All - O'Reilly Media



All of our Ideas and Learning material from all of our topics.



Updated: 2017-05-29T23:13:34Z

 



Four short links: 29 May 2017

2017-05-29T11:40:00Z

Formal Correctness, Conversational Maxims, Learn Datalog, and AlphaGo Retires

  1. An Empirical Study on the Correctness of Formally Verified Distributed Systems (Paper a Day) -- the formal verification ensured the protocol was bug free. By far, the biggest group of bugs relates to assumptions about the behaviour of components that the formally verified system interacts with. These bugs manifest in the interface (or shim layer) between the verified and non-verified components. Buffers, escaping, incomplete reads, unreliable communications, all tripped them up.
  2. Google's Three Secrets to Designing Perfect Conversations -- One secret to making sure that line one leads to two, and two leads to three, comes from James Giongola, creative lead on conversation design and voice direction at Google. He recommends that chat designers take advantage of the rules baked into the Cooperative Principle, a concept created by British philosopher Paul Grice in the 1970s. Grice theorized that people employ all sorts of norms (which are known as Grice’s Maxims) to make sure that conversations flow normally. These maxims serve as simple hacks for anyone writing robo-conversations—the key is to make sure your bot is always offering enough information to keep a conversation going.
  3. Learn Datalog Today -- Learn Datalog Today is an interactive tutorial designed to teach you the Datomic dialect of Datalog. Datalog is a declarative database query language with roots in logic programming. Datalog has similar expressive power as SQL. Prolog as a query language, more or less, designed to parallelize, so popular with the Big Data kids. This is gentler info than reading the Datalog papers.
  4. AlphaGo "Retires" -- DeepMind will release the data from 50 games of the AI playing against itself for the Go community to study. DeepMind is also working on a teaching tool based on AlphaGo to be released sometime in the future. Ke Jie will collaborate with DeepMind on the tool, which Hassabis says should give “all players and fans the opportunity to see the game through the lens of AlphaGo.” I certainly hope this happens. It seems cheap to pop up and win the title, then retire and never play again. IBM's Deep Blue team did this after their software beat Kasparov. If it means the humans never get to learn how to beat the software, then this feels like the engineers using the game rather than loving the game. If you loved the game, you'd leave the game better than you found it rather than unsettled and unresolved.

Continue reading Four short links: 29 May 2017.

(image)



Intelligent Bits: 26 May 2017

2017-05-26T11:00:00Z

(image)

Sukiyaki in French style, brick-and-mortar conversion tracking, route-based pricing, and technological productivity.

  1. Deep recipe transfer — Many recent results have shown the ability to transfer visual patterns and styles across images. Now researchers demonstrate how neural nets can adapt recipes to the culinary styles of particular geographic regions.
  2. Brick, mortar, and bucks — Google can now associate digital ad campaigns with in-store visits and sales by applying machine learning to its wealth of user data, including geolocation, search history, web browsing, app interactions, and now credit card transaction records.
  3. How much for that ride? — Uber applies machine learning to route-based pricing in an effort to become more sustainable by predicting how much you’re willing to pay.
  4. Phew, false alarm — Contrary to popular outcry about technological dislocation of labor, this think tank argues that more innovation is needed to drive productivity and, therefore, jobs.

Continue reading Intelligent Bits: 26 May 2017.

(image)



Four short links: 26 May 2017

2017-05-26T10:30:00Z

Service Availability, Data Share, Eventual Consistency Explained, and Reproducible Deep Learning

  1. The Calculus of Service Availability -- A service cannot be more available than the intersection of all its critical dependencies. If your service aims to offer 99.99% availability, then all of your critical dependencies must be significantly more than 99.99% available. Internally at Google, we use the following rule of thumb: critical dependencies must offer one additional 9 relative to your service—in the example case, 99.999% availability—because any service will have several critical dependencies, as well as its own idiosyncratic problems. This is called the "rule of the extra 9."
  2. datproject -- open source crypto—guaranteed distributed data share, designed for versioned data sets.
  3. How Your Data is Stored -- eventual consistency VERY LUCIDLY explained. It follows the original (entertaining) paper by Leslie Lamport but spells everything out clearly for non-computer-scientists.
  4. OpenAI Baselines -- open source implementations of the interesting published algorithms in deep learning. The papers often gloss over some of the details, so a full and working implementation truly lets others build on research. It's like the reproducibility project for deep learning.

Continue reading Four short links: 26 May 2017.

(image)



Running a word count application using Spark

2017-05-26T10:00:00Z

(image)

How to use Apache Spark’s Resilient Distributed Dataset (RDD) API.

Continue reading Running a word count application using Spark.

(image)



Is finance ready for AI?

2017-05-25T14:00:00Z

(image)

Aida Mehonic explores the role artificial intelligent might play in the financial world.

Continue reading Is finance ready for AI?.

(image)



Using AI to create new jobs

2017-05-25T14:00:00Z

(image)

Tim O’Reilly delves into past technological transitions, speculates on the possibilities of AI, and looks at what's keeping us from making the right choices to govern our creations.

Continue reading Using AI to create new jobs.

(image)



Peeking into the black box: Lessons from the front lines of machine-learning product launches

2017-05-25T14:00:00Z

(image)

Grace Huang shares lessons learned from running and interpreting machine-learning experiments.

Continue reading Peeking into the black box: Lessons from the front lines of machine-learning product launches.

(image)



Accelerate analytics and AI innovations with Intel

2017-05-25T14:00:00Z

(image)

Ziya Ma outlines the challenges for applying machine learning and deep learning at scale and shares solutions that Intel has enabled for customers and partners.

Continue reading Accelerate analytics and AI innovations with Intel.

(image)



Lessons from piloting the London Office of Data Analytics

2017-05-25T14:00:00Z

(image)

Eddie Copeland explores how the London Office of Data Analytics overcame the barriers to joining, analyzing, and acting upon public sector data at city scale.

Continue reading Lessons from piloting the London Office of Data Analytics.

(image)



Data science and deep learning in retail

2017-05-25T11:10:00Z

(image)

The O’Reilly Data Show Podcast: Jeremy Stanley on hiring and leading machine learning engineers to build world-class data products.

In this episode of the Data Show, I spoke with Jeremy Stanley, VP of data science at Instacart, a popular grocery delivery service that is expanding rapidly. As Stanley describes it, Instacart operates a four-sided marketplace comprised of retail stores, products within the stores, shoppers assigned to the stores, and customers who order from Instacart. The objective is to get fresh groceries from popular retailers delivered to customers in a timely fashion. Instacart’s goals land them in the center of the many opportunities and challenges involved in building high-impact data products.

Continue reading Data science and deep learning in retail.

(image)



Travis Lowdermilk and Jessica Rich on building a customer-driven culture at Microsoft

2017-05-25T10:50:00Z

(image)

The O’Reilly Design Podcast: What makes healthy teams healthy, being customer obsessed, and design and research at Microsoft.

This week, I sit down with Travis Lowdermilk senior UX designer at Microsoft, and Jessica Rich, UX researcher at Microsoft; Lowdermilk and Rich are also co-authors of the Customer Driven Playbook. We talk about why failing fast is not always a good approach, sensemaking, and never losing track of the customer’s voice.

Continue reading Travis Lowdermilk and Jessica Rich on building a customer-driven culture at Microsoft.

(image)



JupyterLab: The evolution of the Jupyter web interface

2017-05-25T10:45:00Z

Project Jupyter co-founder Brian Granger on the JupyterLab project, its potential role in scientific and tech communities, and the expanding role of notebooks. Brian Granger is an associate professor of physics at Cal Poly San Luis Obispo, where he also teaches in the university’s undergraduate data science program. His primary area of research is interactive computing with data and, specifically, Jupyter. He started the original IPython Notebook in 2011 and is one of the co-founders of Project Jupyter. He is also an active contributor to and co-leader of the JupyterLab project, which aims to take the Jupyter Notebook interface to the next level with its flexible building blocks for interactive and collaborative computing. We recently discussed JupyterLab, how feedback from notebook users informed its design, how its features could benefit the scientific and technical computing communities, and the role of notebooks in academia and even data journalism. What is JupyterLab and how does it represent what you've called the “evolution of the Jupyter web interface?” The classic Jupyter Notebook offers a number of different building blocks for interactive computing: the notebook, file browser, text editor, terminal, outputs, etc. We view JupyterLab as the evolution of the classic notebook, as it allows a more flexible and powerful way for working with those same building blocks. From a user’s perspective, we hope that everything in JupyterLab is familiar, but even more delightful and productive to work with. Before going further, I want to acknowledge the incredible team working on Jupyter on many fronts, including software, design and organizational aspects. In particular, individuals on the Jupyter Steering Council provide the foundation of the project and are all long-term contributors, without whom the project wouldn’t exist. The team building JupyterLab and PhosphorJS are the driving force behind the work I discuss here. Lastly, we are all grateful to Fernando Perez, creator of IPython and co-founder with us on Jupyter, for setting off into the uncharted wilderness of scientific open source software in 2001. What need is JupyterLab fulfilling among Jupyter Notebook users, particularly those who work in data science or scientific/technical computing? Since the IPython Notebook came out in 2011, we have spent a lot of time talking to individual and organizational users to understand what delights them about the notebook, and what remains really painful. Some of those pain points are specific to the Jupyter Notebook and others are more general challenges they face working with code and data. Also, in 2015, we worked with IBM to run a user experience survey (the results of which are posted on GitHub). Based on all of this feedback, three key factors led us to develop JupyterLab. First, users love the notebook experience, and want it to improve, but without losing the core characteristics that make it the Jupyter notebook. This is important, because we could have re-thought the entire notebook abstraction itself in JupyterLab—but we didn’t. Thus, while we are making improvements to the notebook UI/UX in JupyterLab, notebooks are essentially the same (in fact, it is still 100% the same document format and server). Second, users want to be able to combine, remix, and integrate the different building blocks to better support their workflows. A classic usage pattern we see is data scientists who begin working in an interactive Python shell, then migrate to a notebook, and eventually build and deploy a service based on that code. In the classic notebook, those transitions are really painful. In JupyterLab, we are trying to address the pain points of such an evolving workflow. For exam[...]



Jason Laska and Michael Akilian on using AI to schedule meetings

2017-05-25T10:30:00Z

(image)

The O’Reilly Bots Podcast: The technical and social dynamics of solving scheduling problems.

In this episode of the O’Reilly Bots Podcast, Pete Skomoroch and I talk to Jason Laska and Michael Akilian of Clara Labs, creator of a virtual assistant—Clara—that schedules meetings and interacts in natural language through email.

E-mail is, to me, a highly promising (and somewhat underrated) venue for bots. Messaging is growing quickly, but e-mail is still the standard way to communicate within businesses and especially between businesses. E-mail conventions are somewhat standardized, and much of it is highly routinized—automatically generated reports, receipts, etc.—so it’s ripe for automation.

Laska, who leads the machine learning efforts at Clara Labs, and Akilian, the company’s co-founder and CTO, talk about the reality of developing an AI-driven product, and explain Clara’s human-in-the-loop system. “People are still there to do some of the most challenging aspects of this work, and that’s exactly what you want to use people for,” says Laska.

Continue reading Jason Laska and Michael Akilian on using AI to schedule meetings.

(image)



Four short links: 25 May 2017

2017-05-25T10:20:00Z

Crypto vs. Regulation, Crippling Genomic Research, There Are Bots, and Web Security

  1. Chaffinch -- crypto system that's an interesting response to an attempt to regulate crypto. The Chaffinch system allows several further messages to be steganographically concealed behind the main message. This allows cover traffic to be divulged to any authorities who wish to inspect the confidential information, without compromising the hidden material. The system is evaluated not only in terms of the traditional threat to confidentiality, eavesdroppers with significant computing power, but also in terms of its interaction with the U.K.'s Regulation of Investigatory Powers (RIP) Act, one of the first laws to attempt to engage with cryptography.
  2. We’re About to Cripple the Genomic Medical Era (DJ Patil) -- When we were developing the Precision Medicine Initiative and meeting with Americans across the country, a key concern was ensuring that their data couldn’t be used against them or their families (this is genetic information, so if you share a biological basis, you have overlap in the data). If there is any threat of this data being used in a way that is contrary to research, my deep fear is that people won’t be willing to donate their data. And there are too many people who have diseases who need us to donate our data to help.
  3. There Are Bots, Look Around (Renee DiResta) -- Something very similar happened in finance with the advent of high-frequency trading (the world I came from as a trader at Jane Street): technology was used to distort information flows and access in much the same way it is now being used to distort and game the marketplace of ideas. The future arrived a lot earlier for finance than for politics.
  4. Web Developer Security Checklist -- This checklist is simple, and by no means complete. I’ve been developing secure web applications for over 14 years, and this list contains some of the more important issues that I’ve painfully learned over this period. I hope you will consider them seriously when creating a web application.

Continue reading Four short links: 25 May 2017.

(image)



What Kaggle has learned from almost a million data scientists

2017-05-24T14:00:00Z

(image)

Anthony Goldbloom shares lessons learned from top performers in the Kaggle community and explores the types of machine-learning techniques typically used.

Continue reading What Kaggle has learned from almost a million data scientists.

(image)



The science of visual interactions

2017-05-24T14:00:00Z

(image)

Miriam Redi investigates how machine learning can detect subjective properties of images and videos, such as beauty, creativity, and sentiment.

Continue reading The science of visual interactions.

(image)



Highlights from Strata Data Conference in London 2017

2017-05-24T14:00:00Z

Watch highlights covering data-driven business, data engineering, machine learning, and more. From Strata Data Conference in London 2017.Experts from across the data world came together in London for Strata Data Conference. Below you'll find links to highlights from the event. Using AI to create new jobs Tim O’Reilly delves into past technological transitions, speculates on the possibilities of AI, and looks at what's keeping us from making the right choices to govern our creations. Watch "Using AI to create new jobs." The science of visual interactions Miriam Redi investigates how machine learning can detect subjective properties of images and videos, such as beauty, creativity, and sentiment. Watch "The science of visual interactions." Machine learning is a moonshot for us all Darren Strange asks: What part will we each play in what is sure to be one of the most exciting times in computer science? Watch "Machine learning is a moonshot for us all." What Kaggle has learned from almost a million data scientists Anthony Goldbloom shares lessons learned from top performers in the Kaggle community and explores the types of machine-learning techniques typically used. Watch "What Kaggle has learned from almost a million data scientists." Another one bytes the dust Using the music industry as an example, Paul Brook shows how modern information points bring new data that changes the way an organization will make decisions. Watch "Another one bytes the dust." The data subject first? Aurélie Pols draws a broad philosophical picture of the data ecosystem and then hones in on the right to data portability. Watch "The data subject first?" Real-time intelligence gives Uber the edge M. C. Srivas covers Uber's big data architecture and explores the real-time problems Uber needs to solve to make ride sharing smooth. Watch "Real-time intelligence gives Uber the edge." Lessons from piloting the London Office of Data Analytics Eddie Copeland explores how the London Office of Data Analytics overcame the barriers to joining, analyzing, and acting upon public sector data at city scale. Watch "Lessons from piloting the London Office of Data Analytics." Accelerate analytics and AI innovations with Intel Ziya Ma outlines the challenges for applying machine learning and deep learning at scale and shares solutions that Intel has enabled for customers and partners. Watch "Accelerate analytics and AI innovations with Intel." Is finance ready for AI? Aida Mehonic explores the role artificial intelligent might play in the financial world. Watch "Is finance ready for AI?" Peeking into the black box: Lessons from the front lines of machine-learning product launches Grace Huang shares lessons learned from running and interpreting machine-learning experiments. Watch "Peeking into the black box: Lessons from the front lines of machine-learning product launches." Continue reading Highlights from Strata Data Conference in London 2017.[...]



The data subject first?

2017-05-24T14:00:00Z

(image)

Aurélie Pols draws a broad philosophical picture of the data ecosystem and then hones in on the right to data portability.

Continue reading The data subject first?.

(image)



Machine learning is a moonshot for us all

2017-05-24T14:00:00Z

(image)

Darren Strange asks: What part will we each play in what is sure to be one of the most exciting times in computer science?

Continue reading Machine learning is a moonshot for us all .

(image)



Another one bytes the dust

2017-05-24T14:00:00Z

(image)

Using the music industry as an example, Paul Brook shows how modern information points bring new data that changes the way an organization will make decisions.

Continue reading Another one bytes the dust.

(image)



Real-time intelligence gives Uber the edge

2017-05-24T14:00:00Z

(image)

M. C. Srivas covers Uber's big data architecture and explores the real-time problems Uber needs to solve to make ride sharing smooth.

Continue reading Real-time intelligence gives Uber the edge.

(image)



12 qualities of effective design organizations

2017-05-24T11:00:00Z

Learn what it takes to make the most of your design team.In 2011, things began to turn around for NBA basketball team the Golden State Warriors. After years of apathetic leadership and poor performance, new owners made big moves, trading perceived franchise player Monta Ellis for Andrew Bogut and hiring coach Mark Jackson. The next couple of years saw more talent acquisition, such that by 2013–2014, the team had its core roster: Stephen Curry, Klay Thompson, Harrison Barnes, Draymond Green, Andre Iguodala, and Andrew Bogut. And even though they won 50 games for the first time in 20 years, they lost in the first round of the playoffs. Internal turmoil tore up the coaching staff, as Mark Jackson’s approach seemed to be one of “my way or the highway”—assistants who disagreed with him were let go, and players were pitted against each other. After the playoffs, Mark Jackson was fired, to be replaced by Steve Kerr, who installed a new coaching staff, and a much more inclusive and joyful management style that welcomed respectful disagreement in search of the best answer. Under such leadership, and with no significant roster changes, the Warriors dominated the league, winning 67 games and the NBA championship. There are two lessons for any team. The first is that skill and talent matter. By making big roster moves, the Warriors made back-to-back playoffs for the first time in over 20 years. But talent isn’t sufficient. The second lesson is that to get the most out of a team requires sensitive management, visionary leadership, and well-run operations. Design teams often suffer in this second area because, compared with other corporate functions like engineering and marketing, design is newer and its appreciation is less sophisticated. This nascency means that (a) most people in an organization have never worked with a truly effective design team, and (b) most designers haven’t been part of fully actualized teams, and so they don’t know what they need in order to realize their own potential. Many design teams have the raw talent to realize the expanded role outlined in the prior chapter, but don’t yet have the maturity to embrace it. A design team’s output is the result not only of their skill, but the sophistication and sensitivity of how they operate. In this chapter, we present a set of qualities of effective design organizations. Assessing a team’s performance against each of these qualities clarifies opportunities for improvement. The qualities are broken up into three groups: Foundation, Output, and Management (Table 1-1). The Foundation outlines the core concepts that drive the team’s behavior, and explain its very reason for being. With a strong Foundation established, energies then shift toward Output and Management. These qualities are indicative of the broader creative/operational split that is required to sustainably deliver good design, and is a theme throughout this book. Output and Management need to be tackled in tandem, as they reinforce each other. Output addresses what most people think of when considering design—is the team able to produce quality work across the necessary set of capabilities? Management addresses the unsung and often overlooked aspects of actually running a team. To realize team longevity and continued broadening impact, it’s imperative to treat operations as seriously as the work product. Table 1-1. Table 1-1. The 12 qualities of effective design organizations Foundation Output Management[...]



Kelly Shortridge on overcoming common missteps affecting security decision-making

2017-05-24T10:35:00Z

(image)

The O’Reilly Security Podcast: How adversarial posture affects decision-making, how decision trees can build more dynamic defenses, and the imperative role of UX in security.

In this episode, I talk with Kelly Shortridge, detection product manager at BAE Systems Applied Intelligence. We talk about how common cognitive biases apply to security roles, how decision trees can help security practitioners overcome assumptions and build more dynamic defenses, and how combining security and UX could lead to a more secure future.

Continue reading Kelly Shortridge on overcoming common missteps affecting security decision-making.

(image)



Four short links: 24 May 2017

2017-05-24T10:00:00Z

Travel Mode, Justice Data, Threat Dragon, and Voice Editing

  1. 1Password Travel Mode -- enable Travel Mode, and all devices not marked Safe For Travel are deleted from your devices. There's no indicator to let border agents know that Travel Mode is enabled.
  2. Measures for Justice -- collects data on justice systems in several states, funded by Gates and Zuckerberg foundations. As Bach says, “Justice in America happens in 3,000 counties, each with its own justice system.” (via Wired)
  3. OWASP Threat Dragon -- open source threat modeling tool from OWASP. It can be used as a standalone desktop app for Windows, MacOS, and Linux or as a web application. (via Tech Beacon)
  4. More Detail on Adobe's Voice Editing Software -- VoCo is based on an optimization algorithm that searches the voice recording and chooses the best possible combinations of phonemes (partial word sounds) to build new words in the user’s voice. To do this, it needs to find the individual phonemes and sequences of them that stitch together without abrupt transitions. It also needs to be fitted into the existing sentence so that the new word blends in seamlessly. Words are pronounced with different emphasis and intonation depending on where they fall in a sentence, so context is important. [...] In case the synthesized word isn’t quite right, VoCo offers users several versions of the word to choose from. The system also provides an advanced editor to modify pitch and duration, allowing expert users to further polish the track.

Continue reading Four short links: 24 May 2017.

(image)



Reliability with Kafka

2017-05-24T10:00:00Z

(image)

Five questions for Gwen Shapira about how Kafka can enable business agility.

I recently sat down with Gwen Shapira, system architect at Confluent, to talk about Kafka—why it’s becoming popular (particularly with financial organizations), the benefits it can have for your organization, and how to adopt it safely and reliably. Here are some highlights from our talk.

What are some of the more exciting use cases for Kafka?

I’d say it’s most exciting to see how Apache Kafka use cases have evolved over the last few years.

Continue reading Reliability with Kafka.

(image)



5 things to learn before learning React

2017-05-24T10:00:00Z

(image)

Level up your skills set before diving into React.

If you haven’t built a single-page application before, React really forces you to level-up your skill set. React as a library has a lot to offer - you can build shareable components that have a clear flow of data (which makes debugging much, much easier). Many times, React just lets you write JavaScript to get your work done.

For the last year or so, I've been writing apps with React and Redux, and working at a coding bootcamp part-time, where I help people learn React and Redux. After watching people struggle with out-of-date tutorials and blog posts, and the JavaScript ecosystem in general, I compiled a list of things you might want to know BEFORE you start learning React. The list below may sound exhaustive, but once you feel fluent in these concepts and skills, it will make building your first app in React easier and more fun.

Continue reading 5 things to learn before learning React.

(image)



Four short links: 23 May 2017

2017-05-23T10:05:00Z

TensorFlow Cookbook, Welcoming Newcomers, Patent Win, and Face Classification

  1. TensorFlow Cookbook Code -- code from Nick McClure's TensorFlow Machine Learning Cookbook.
  2. How I Welcome Newcomers (Dan Meyer) -- he has a Chrome extension to highlight newbie tweeters in math chats, so he can give them a warm welcome. What a great idea!
  3. Supreme Court Smacks Down Venue Shopping for Patent Cases -- huzzah, those pricks in the Eastern District of Texas get what they deserve. More than 40% of all patent lawsuits are filed in East Texas. Of those, 90% are brought by "patent trolls," according to a study published in a Stanford Law School journal.
  4. Face Classification -- Real-time face detection and emotion/gender classification using fer2013/imdb data sets with a keras CNN model and openCV.

Continue reading Four short links: 23 May 2017.

(image)



Jupyter Digest: 22 May 2017

2017-05-22T15:35:00Z

(image)

TensorFlow cookbook materials, source notebooks, Python lectures, and Software Carpentry.

  • nfmcclure/tensorflow_cookbook. Nick McClure (@nfmcclure) released all the companion materials for his book Tensorflow Machine Learning Cookbook (published by Packt, which is absolutely crushing it in publishing with Jupyter).

  • Probably Overthinking It source notebooks. If you're not a regular reader of Allen Downey's blog Probably Overthinking It, then you're missing out. He regularly produces a thought-provoking series on stats, programming, and math. But, did you know you can also get his source articles as Jupyter Notebooks? Well, neither did I, until I signed on to doing this weekly series.

  • Scientific Python Lectures. Robert Johansson's (@rjohansson on GitHub) series of notebooks on various scientific computing topics. Are you looking to do Fortran and C, or high-performance computing? Also, for extra credit, he's the author of Numerical Python, which you can find in Safari.

  • Software Carpentry. If you're an organization looking to get started with the whole Python stack, or just software engineering in general, then you should check out Software Carpentry. They organize a series of courses across the world on topics like Python, Git, R, the command line, Unix. You name, they've got it.

Continue reading Jupyter Digest: 22 May 2017.

(image)



Four short links: 22 May 2017

2017-05-22T10:05:00Z

Meta Tutorial, Network Game, Indigenous VR, and Facebook Moderation

  1. An Interactive Tutorial on Making Interactive Tutorials -- full of the little details that you only learn by doing many times. (via @redblobgames)
  2. Netsim -- a simulator game designed to teach high schoolers about networking theory. (via @errorinn)
  3. Indigenous Australia in VR (SMH) -- The idea is to create a complex game where the user, wearing an Oculus Rift virtual reality headset, can engage with and learn about Aboriginal culture—and it's the campus elders and other Indigenous people who are driving the content.
  4. The Facebook Files -- the Guardian has copies of some of Facebook's moderation docs. Eye-wateringly, and eye-openingly, comprehensive guides to the situations that crop up online and the rules for navigating them. In one of the leaked documents, Facebook acknowledges “people use violent language to express frustration online” and feel “safe to do so” on the site. It says: “They feel that the issue won’t come back to them and they feel indifferent toward the person they are making the threats about because of the lack of empathy created by communication via devices as opposed to face to face. There's a lot of (perhaps amateur) psychological analysis behind these guidelines because of the complex social and personal circumstances in the edge-cases and conflicts. The big challenge for Facebook is to curtail some behaviour without removing the engagement-driving illusion that it's "my Facebook" that I am posting to (when, in fact, it might be more accurate to refer to posting as "crapping all over my friends' screens.")

Continue reading Four short links: 22 May 2017.

(image)



Intelligent Bits: 19 May 2017

2017-05-19T10:45:00Z

(image)

AutoML, AI photo editing, AI product studio, and Apple and dark data.

  1. AutoML — The Google Brain team successfully applied reinforcement learning techniques to auto-design optimal neural net architectures.
  2. AI-powered photo editing — The hits just on keep coming with AI for photo editing applications. Alyosha Efros’s research group at Berkeley helps lead the charge, following up on their popular CycleGAN work with their new deep colorization results.
  3. All Turtles — Phil Libin, former Evernote CEO, has left General Catalyst to start a studio for helping founders build AI-based products.
  4. Apple wants dark data — Structuring and making use of dark or unstructured data remains a big challenge even as artificial intelligence technologies take off. Apple has acquired Lattice to address that need, perhaps leaving the door open for another startup to emerge and serve the broader market with similar capabilities.

Continue reading Intelligent Bits: 19 May 2017.

(image)



Four short links: 19 May 2017

2017-05-19T10:25:00Z

Algorithmic Fallibility, AI Sketches, Traffic Obfuscation, and Engineer-Manager Pendulum

  1. Algorithmic Fallibility and Economic Organization -- algorithms have benefits (when they get the right answer) and costs (when they get the wrong answer). This article creates three scenarios and uses the tools of economics to analyze them.
  2. Google Releases Sketches -- Sketch-RNN, a generative model for vector drawings, is now available in Magenta. Comes with 50M drawings as training data.
  3. Bedlam -- Google Chrome extension to generate random web traffic/DNS requests to make your web traffic data less valuable for selling.
  4. The Engineer-Manager Pendulum (Charity Majors) -- The best frontline eng managers in the world are the ones who are never more than 2-3 years removed from hands-on work, full time down in the trenches. The best individual contributors are the ones who have done time in management.

Continue reading Four short links: 19 May 2017.

(image)



Activating human intrusion detection systems

2017-05-19T10:00:00Z

(image)

Expanding the blue team by building a security culture program.

Continue reading Activating human intrusion detection systems.

(image)



Exploring progressive web apps in the real world

2017-05-19T10:00:00Z

(image)

How companies are providing native experiences in their mobile apps.

Continue reading Exploring progressive web apps in the real world.

(image)



How to write a sound hypothesis when conducting user research

2017-05-18T11:15:00Z

(image)

Cindy Alvarez outlines the components of a hypothesis and shares examples of successful and unsuccessful hypotheses.

Continue reading How to write a sound hypothesis when conducting user research.

(image)



How AI is used to infer human emotion

2017-05-18T11:00:00Z

Rana el Kaliouby discusses the techniques, possibilities, and challenges around emotion AI today.Rana el Kaliouby is the co-founder and CEO of Affectiva, an emotion measurement technology company that grew out of MIT's Media Lab. Rana is giving a talk, The science and applications of the emerging field of artificial emotional intelligence, at the Artificial Intelligence Conference in New York City on June 28, 2017. We recently caught up with Rana to discuss the techniques, possibilities, and challenges around emotion AI today. Our conversation has been edited for clarity. For those less familiar with emotion AI, can you describe what the field encompasses? Emotion AI is the idea that devices should sense and adapt to emotions like humans do. This can be done in a variety of ways—understanding changes in facial expressions, gestures, physiology, and speech. Our relationship with technology is changing, as it’s becoming a lot more conversational and relational. If we are trying to build technology to communicate with people, that technology should have emotional intelligence (EQ). This manifests in a broad range of applications: from Siri on your phone to social robots, even applications in your car. How is emotion AI related to sentiment analysis for natural language processing? Social scientists who have studied how people portray emotions in conversation found that only 7-10% of the emotional meaning of a message is conveyed through the words. We can mine Twitter, for example, on text sentiment, but that only gets us so far. About 35-40% is conveyed in tone of voice—how you say something—and the remaining 50-60% is read through facial expressions and gestures you make. Technology that reads your emotional state, for example by combining facial and voice expressions, represents the emotion AI space. They are the subconscious, natural way we communicate emotion, which is nonverbal and which complements our language. What we say is also very cognitive—we have to think about what we are going to say. Facial expressions and speech actually deal more with the subconscious, and are more unbiased and unfiltered expressions of emotion. What techniques and training data do machines use to perceive emotion? At Affectiva, we use a variety of computer vision and machine learning approaches, including deep learning. Our technology, like many computer vision approaches, relies on machine learning techniques in which algorithms learn from examples (training data). Rather than encoding specific rules that depict when a person is making a specific expression, we instead focus our attention on building intelligent algorithms that can be trained to recognize expressions. Through our partnerships across the globe, we have amassed an enormous emotional database from people driving cars, watching media content, etc. A portion of the data is then passed on to our labeling team, who are certified in the Facial Action Coding System (FACS). Their day-to-day job is to take video from a repository and label it as training data for the algorithms. We are continuously investing in[...]



Paris Buttfield-Addison on what’s new in Swift programming

2017-05-18T10:45:00Z

(image)

The O’Reilly Programming Podcast: Applying the latest language features to build video games and containerized microservices.

In this episode of the O’Reilly Programming Podcast, I talk about Swift with Paris Buttfield-Addison, co-founder of Secret Lab, a mobile development studio that builds games and apps for mobile devices. He is the co-author of Learning Swift, and a presenter of the Learning Path Getting Started with Swift on the iPad and the video Ultimate Swift Programming.

Continue reading Paris Buttfield-Addison on what’s new in Swift programming.

(image)



Four short links: 18 May 2017

2017-05-18T10:24:00Z

Checking Fact-Checkers, Simpler Java, JSON Feed, and Street-Fighting Mathematics

  1. Checking How Fact-checkers Check -- I evaluate the performance of two major online fact-checkers, Politfact at Tampa Bay Times and Fact Checker at Washington Post, comparing their interrater reliability using a method that is regularly utilized across the social sciences. I show that fact-checkers rarely fact-check the same statement, and when they do, there is little agreement in their ratings. Approximately, 1 in 10 statements is fact-checked by both fact-checking outlets, and among claims that both outlets check, their factual ratings have a Cohen’s κ of 0.52, an agreement rate much lower than what is acceptable for social scientific coding. The results suggest that difficulties in fact-checking elites’ statements may limit the ability of journalistic fact-checking to hold politicians accountable. (via Marginal Revolution)
  2. Kotlin -- a Swift-like take on Java. Statically typed programming language for modern multiplatform applications 100% interoperable with Java and Android. Steve Yegge loves it, and here's a rundown of the main language features.
  3. JSON Feed -- another tilt at the content syndication windmill. "It's Atom but in convenient COBOL Object Notation," he said twitching. "Both remaining bloggers have signed up to use it!"
  4. Street-Fighting Mathematics (PDF) -- MIT book on the art of educated guessing and opportunistic problem-solving. The major sections are: Dimensions; Easy cases; Lumping; Pictorial proofs; Taking out the big part; Analogy.

Continue reading Four short links: 18 May 2017.

(image)



Using Docker in production

2017-05-18T10:00:00Z

(image)

Five questions for Laura Frank about orchestration, security, and beyond.

I recently sat down with Laura Frank, Docker Captain and director of engineering at Codeship, to discuss the evolution of the Docker ecosystem and how it compares to other orchestration tools. Here are some highlights from our talk.

The Docker ecosystem has evolved rapidly over the past couple of years. How is using Docker now different than it was, say, two years ago?

Right now, Docker is an excellent tool to manage distributed applications. This is the result of quite a bit of evolution; in its earlier stages, Docker focused mainly on managing containers themselves. Thinking back to two or three years ago, getting started with Docker was a bit of a pain because there weren’t very mature developer tools in the ecosystem. Instead you were left with documentation and really long “docker run” commands, and you really had to know what was happening at the container level. Now Docker has grown and evolved a bit to where the container is just an implementation detail, allowing you as an engineer to focus on what’s really important: the services themselves. Orchestration tools like Docker (in Swarm Mode), Kubernetes, and Mesosphere allow you to declare your services once and then run them anywhere using containers. The focus now is more on running highly-available applications and less on the inner workings of the container itself, so you interact with Docker on a different level.

Continue reading Using Docker in production.

(image)



How do I build an API?

2017-05-18T08:00:00Z

(image)

Learn how to build both web and traditional application programming interfaces (APIs).

Continue reading How do I build an API?.

(image)



What is an API?

2017-05-18T08:00:00Z

(image)

Learn the basics of application programming interfaces (APIs), their purpose, and the key concepts underlying the technology.

Continue reading What is an API?.

(image)



How do I use an API?

2017-05-18T08:00:00Z

(image)

Learn how to use different types of application programming interfaces (APIs).

Continue reading How do I use an API?.

(image)



How to do client-side form validation with Elm

2017-05-17T18:20:00Z

(image)

Elm’s static typing and compiler error messages lead to more productivity.

Form validation, taking user entered data and giving them hints about what they need to do, is one of the most common tasks that a web developer encounters. Even though it’s everywhere, it’s easy to forget how nuanced and tricky form validation can be. At the crucial time when a user is signing up for your service, or filling out their profile, it’s easy to irritate a user with incomplete feedback, fields that are deleted upon failed validation, and cryptic error messages.

Form validators are nuanced because every field usually has a different set of error states that could potentially pop up. Emails can be invalid. Passwords need to be a certain length. Terms of service can be, well, not agreed to.

Continue reading How to do client-side form validation with Elm.

(image)



The stages of enterprise IoT adoption

2017-05-17T17:05:00Z

(image)

Teresa Tung on building a business case for the Internet of Things.

Continue reading The stages of enterprise IoT adoption.

(image)



Four short links: 17 May 2017

2017-05-17T11:15:00Z

Shipping Apps, Cloud Economics, Computational Theory, and Imitation Learning

  1. How Etsy Ships Apps -- starts with a nifty summary of their chatops-based push process, then moves to how they tackle shipping for mobile apps. So, we built a vessel that coordinates the status, schedule, communications, and deploy tools for app releases. Here’s how Ship helps: (1) keeps track of who committed changes to a release; (2) sends Slack messages and emails to the right people about the relevant events; (3) manages the state and schedule of all releases.
  2. Usage Patterns and the Economics of the Cloud (Adrian Colyer) -- cloud providers overwhelmingly use static pricing models; what’s going on? Here’s the short summary: the data shows that there is actually very little variation in demand volatility for cloud datacenters at the moment, thus the current pricing model makes sense. If you look more closely at actual CPU utilization rates, though, you see that behind the constantly powered-on VMs, there are true variations in usage patterns. Therefore, as we move to cloud-native applications, and especially to models such as serverless that can much more effortlessly and granularly scale up and down in response to changing demands, we can expect the optimum pricing models to also change. Even then, it appears that having just two price bands, peak and off-peak—with off-peak times set in advance, would obtain the majority of the efficiency gains available.
  3. New Kind of Science -- available free. (via Stephen Wolfram's long article on NKoS and what's happened in the last 15 years).
  4. One-Shot Imitation Learning -- ideally, robots should be able to learn from very few demonstrations of any given task and instantly generalize to new situations of the same task, without requiring task-specific engineering. In this paper, we propose a meta-learning framework for achieving such capability, which we call one-shot imitation learning. (via OpenAI)

Continue reading Four short links: 17 May 2017.

(image)



Augmenting industrial reality

2017-05-17T10:00:00Z

(image)

Integration tools and vision systems represent two developments in enterprise AR that point to bigger things to come.

Augmented Reality is clearly poised to shift everything about our world. Facebook just devoted a whole building, Building 8, to creating AR—they’re currently hiring camera systems architects and neural imaging engineers. Companies like Chevrolet are creating amazing examples of realtime AR—in which cars are digitally rendered so realistically you’d swear they were filmed (nope). The Economist is reporting on the space in a thoughtful and highly accurate way. The implication of all of this is that this is big business. And it is. Pokemon GO was a flash-trend that shone a spotlight on how this can excite people. It also proved that AR can be used to make money. AR is poised to make a big leap in the next five years.

And there’s only one space in which AR has deeply proven its utility in so far. Namely: manufacturing and industry—enterprise.

Continue reading Augmenting industrial reality.

(image)



Four short links: 16 May 2017

2017-05-16T18:50:00Z

Flash Organizations, Collaboration Data Set, De-Anonymizing Mobile Data, and Hacking Economics Flash Organizations: Crowdsourcing Complex Work By Structuring Crowds As Organizations -- Our system introduces two technical contributions: 1) encoding the crowd’s division of labor into de-individualized roles, much as movie crews or disaster response teams use roles to support coordination between on-demand workers who have not worked together before; and 2) reconfiguring these structures through a model inspired by version control, enabling continuous adaptation of the work and the division of labor. We report a deployment in which flash organizations successfully carried out open-ended and complex goals previously out of reach for crowdsourcing, including product design, software development, and game production. Media Manipulation and Disinformation Online (PDF) -- research from Data & Society that seeks to answer the questions: Who is manipulating the media? Where do these actors operate? What motivates media manipulation? What techniques do media manipulators use? Why is the media vulnerable? What are the outcomes?" (via BoingBoing) Trajectory Recovery From Ash (Adrian Colyer) -- how easy it is to deanonymize theoretically anonymous data. Even in a data set in which you might initially think there is no chance of leaking information about individuals, they can recover data about individual users with between 73% and 91% accuracy—even in data sets which aggregate data on tens of thousands to hundreds of thousands of users! Their particular context is mobile location data, but underpinning the discovery mechanism is a reliance on two key characteristics: (1) individuals tend to do the same things over and over (regularity)—i.e., there are patterns in the data relating to given individuals, and (2) these patterns are different across different users (uniqueness). Economia: A Festival on Economy Without the Economists (We Make Money Not Art) -- As curators Wiepko Oosterhuis and Olga Mink wrote: Why not start by treating economics like any other technology? Play with it, hack it, use input from other disciplines, unleash science fiction on it, approach it in an artistic manner. In short, take ownership so that we can reshape and rework economics as we see fit. I love the idea of the minimum wage machine: Turning the crank yielded a one cent euro coin every 4.018 seconds, that’s €8.96 an hour, the minimum wage in The Netherlands right now. The coins dropped as long as you turned the crank. I saw many people trying it. All of them stopped after the first few cents. You want to have a go because it’s a fun and straightforward installation, but you quickly realize how depressing and mind-numbing routine work is. Continue reading Four short links: 16 May 2017.[...]



Architecting actionable insights

2017-05-16T16:10:00Z

(image)

Bas Geerdink details the technology stack for real-time account forecasting at ING, and outlines how Spark is used for outbound communications.

Continue reading Architecting actionable insights.

(image)



The business advantages of embedding analytics into applications

2017-05-16T11:00:00Z

Access to critical data in real time enables workers to generate insights from large amounts of information.Imagine, if you will, that you’re a product manager recently tasked with adding some whiz-bang visual analytics to your software, web site, or mobile app. Your first instinct may be to build your own solution as part of your existing product. However, you have to ask yourself if you have the team resources to accomplish the chore, or if you would rather have your team work to keep your core product fast and functional. Luckily, there is another option, one that leverages embedded analytics. With that in mind, let’s take a deeper dive into the world of embedded analytics and determine why you might choose it, who is doing it, and how to evaluate available off-the-shelf solutions. The rise of the data-driven organization Organizations are clamoring to unlock the value inside the massive volumes of data they are collecting. Data-driven employees make more informed decisions that help companies beat the competition. Employees need applications that help them make sense of all the data available for decision-making. Then they need to explore and analyze the data so they can make the best decisions. The hunger for data and analytics is changing the expectations that workers have for software. Business users expect customizable dashboards and reports as part of every application they use. More importantly, they expect to be able to quickly and easily explore and interact with data sets. Combine that with the growing complexity, size, and variation in data sets, and software developers are facing increasingly demanding challenges in helping their users and customers gain the insights they need. Modern data platforms We live in a time where the speed and availability of multitudes of data is unprecedented. Every data set can be tapped, combined with others, and analyzed to yield business insights. The challenge of quickly providing powerful insight is compounded by the rapidly developing modern data stores that have been built to handle the dramatic increase in volume, velocity and variety of data -- billions of rows, fast streaming data, and often unstructured textual and document data. There are NoSQL databases like Cassandra, HBase, and MongoDB; data processing frameworks like Hadoop and Spark; query engines like Impala and Hive; stream processing tools like Storm and Kafka; and text indexing frameworks like ElasticSearch and Solr. Cloud providers have entered the fray with scalable and inexpensive datastores such as Amazon’s Redshift, Google’s BigQuery and Spanner, and Microsoft’s HDInsight. As you can see, building your own visual analytics within your product could quickly strain architectural and developer resources, and their [...]



How do I package my Java application as a Docker image using Gradle?

2017-05-16T08:00:00Z

(image)

Learn how to use Gradle, a popular build tool for Java developers, to package your Java application as a Docker image and run as a Docker container.

Continue reading How do I package my Java application as a Docker image using Gradle?.

(image)



How do I create Docker images and run containers using Maven?

2017-05-16T08:00:00Z

(image)

Learn how to use Apache Maven, a project management and comprehension tool, to package your Java application as a Docker image and run as a Docker container.

Continue reading How do I create Docker images and run containers using Maven?.

(image)



How can I deploy a multi-container application with Docker Compose?

2017-05-16T08:00:00Z

(image)

Learn how to use Docker to distribute multi-container applications into a standardized unit for seamless software development.

Continue reading How can I deploy a multi-container application with Docker Compose?.

(image)



Get introduced to the new Java® 9 Platform Module System (JPMS) with Paul Deitel

2017-05-15T17:00:00Z

This two-day live online training covers JPMS, a tool Paul Deitel calls “the most significant new Java software-engineering technology since the inception of Java.”Paul Deitel, CEO of Deitel & Associates, Inc., co-author of Java® 9 for Programmers, and an Oracle Java Champion, is leading a live online introduction to the Java Platform Module System (JPMS). Deitel’s two-day live online training will be presented only on Safari, O’Reilly’s learning platform. Modularity—the result of Project Jigsaw—benefits developers at all levels by helping them manage and reduce complexity, leading to increased productivity in the construction, maintenance, and evolution of software systems—especially large-scale software systems. What you’ll learn in Introduction to Modularity with the Java® 9 Platform Module System In this live online training, you’ll: Understand the motivation for modularity. Review key JPMS requirements. Modularize pre-Java-9 code, then execute the modularized version. Create module declarations that specify module dependencies with requires, and specify the packages a module makes available to other modules with exports. Selectively allow runtime reflection of types with open and opens. Use services to loosely couple system components, making large-scale systems easier to develop and maintain. Indicate that a module explicitly uses a service or provides a service implementation with uses and provides...with, respectively. Use the jdeps command to determine a module’s dependencies. Understand how unnamed and automatic modules enable you to continue using non-modularized pre-Java-9 code in Java 9. Use the NetBeans IDE to create module graphs to visualize the modular JDK as well as your own modular systems. See the module resolver in action as it determines runtime dependencies. Use jlink to create smaller runtimes appropriate for resource-constrained devices—for example, Internet of Things (IoT) devices. And more. Visit Safari for the detailed course outline and prerequisites for Introduction to Modularity with the Java® 9 Platform Module System. How Java 9 modularity will make developers’ lives easier According to JSR 376: Java Platform Module System, the key goals of modularizing the Java SE Platform are: Reliable configuration—You must explicitly declare dependencies between modules in a manner that’s recognized both at compile time and execution time. The system can automatically walk through these dependencies (via the module resolver) to determine the subset of all modules required to support your app. Strong encapsulation—The packages in a m[...]



Jupyter Digest: 15 May 2017

2017-05-15T11:00:00Z

TSFRESH, 100 days of algorithms, how JupyterHub tamed big science, colorizing photos. TSFRESH. TSFRESH is a "time series feature extraction based on scalable hypothesis tests." In layman's terms, it finds interesting things on a time-series chart for you automatically. The notebooks folder has Jupyter examples that show how to use it in your work, like this one that uses accelerometer data to figure out when you're walking, climbing stairs, or just doing nothing at all. (Submitted anonymously.) 100 Days of Algorithms. Tomáš Bouda (@coells on GitHub) compiles a nice list of examples that illustrate a host of different algorithms with Python. If a title like "Day 14 - huffman codes.ipynb" lights you up, then you're gonna love this (times 100). Also, bravo for not calling it "50 algorithms to whiteboard before you die." How JupyterHub tamed big science: Experiences deploying Jupyter at a supercomputing center. If you're trying to deploy Jupyter at scale in your org, then this session at the upcoming JupyterCon (Aug 22-23 in NYC) will be the place to be to learn how. Interactive Deep Colorization. Richard Zhang, a PhD candidate at UC Berkeley, uses this notebook to illustrate a technique he's developed to colorize black and white photos (see below). Take that, Dorothea Lang.   allowfullscreen frameborder="0" height="315" src="https://www.youtube.com/embed/eL5ilZgM89Q" width="560">   Continue reading Jupyter Digest: 15 May 2017.[...]



Four short links: 15 May 2017

2017-05-15T10:35:00Z

Formal Systems, Deep Learning, Assembly Games, and Logs vs. Metrics Form and Content in Computer Science (Marvin Minsky) -- Minsky's 1970 ACM Turing Lecture. Let us consider a more elementary, but still puzzling, trade-off, that between addition and multiplication. How many multiplications does it take to evaluate the 3 X 3 determinant? If we write out the expansion as six trinomials, we need 12 multiplications. If we collect factors, using the distributive law, this reduces to nine. What is the minimum number, and how does one prove it, in this and in the n X n case? The important point is not that we need the answer. It is that we do not know how to tell or prove that proposed answers are correct! The interesting work currently being done in formal systems has a long heritage, but struggled for attention and interest in researchers for a long time. Questions & Intuition for Tackling Deep Learning Problems -- a great list. Never mind a neural network; can a human with no prior knowledge, educated on nothing but a diet of your training data set, solve the problem? Is your network looking at your data through the right lens? Is your network learning the quirks in your training data set, or is it learning to solve the problem at hand? Does your network have siblings that can give it a leg-up (through pre-trained weights)? Is your network incapable or just lazy? If it’s the latter, how do you force it to learn? Computer Games that Make Assembly Language Fun (IEEE Spectrum) -- three polished games that do a surprisingly good job of making coding in assembly language fun. To be clear, none of these titles involve writing assembly for real hardware. They all use virtual systems with minimal instruction sets. Still, they do capture the essence of assembly coding, with complex behaviors squeezed out of simple commands. Logs vs. Metrics -- difference between logs and metrics is huge. A log is an immutable record of discrete events that happened over time while metrics are a set of numbers that give information about a particular process or activity usually recorded over time to form a time series. I loved the RED method: "request rate, error rate, and duration of requests to tell you how busy your service is, whether there are any errors in it, and what its latency is." Continue reading Four short links: 15 May 2017.[...]



Understanding the Kubernetes ecosystem

2017-05-15T10:00:00Z

Five Questions for Sebastien Goasguen about Kubernetes and the cloud native tools that support it.I recently sat down with Sebastien Goasguen, Senior Director of Cloud Technologies at Bitnami, to talk about Kubernetes and its ecosystem—why the tool is becomingly increasingly popular, the tools you can use to support it, and how to adopt containerized architectures at your organization. Here are some highlights from our talk. Why is Kubernetes so popular? I am a big believer that you need to try things yourself to form an opinion about them, but many people who try Kubernetes like it right away. The system is not really complicated or different from other cluster management systems; what’s surprising is how difficult it is to kill your application after you’ve deployed it. There are many built-in levels of fault tolerance that make your application resilient and scalable. If you like APIs and solid clients, you will appreciate the Kubernetes API as well. So the bottom line is: try it, and I bet you will be hooked. What other tools work well with Kubernetes? Google is really investing for the long term with Kubernetes; they realized right away that they needed to build a true ecosystem of partners if the project was to be a success. They ended up donating it to the Cloud Native Foundation (CNCF), a non-profit foundation hosted by the Linux foundation with the aim to help with governance, awareness and community activities. Now CNCF has grown to encompass more cloud native software projects like Prometheus, Fluentd, OpenTracing, and Containerd. We can expect other projects to join in the future. That means that naturally—through a sort of “CNCF osmosis”—Prometheus is used with Kubernetes for monitoring, Fluentd for log aggregation. In addition, Kubernetes has its own incubator where you can find projects like kompose, kargo, and Helm, which recently graduated from the incubator. That said, there are also non-CNCF tools, such as Ansible and Terraform, that work well with Kubernetes, but you can expect a natural affinity between the various CNCF projects. What are some unresolved challenges with the Kubernetes project? There is a challenge with the speed of innovation and scale of the project, both from a technical and governance standpoint. There are multiple efforts to break up the main repository to help speed up pull request reviews and bring attention to issues. There are currently over 5,000 issues and over 600 pull requests. The project needs to find a way to sustain momentum, allow new contributors to help, and at the same time maintain cod[...]



Intelligent Bits: 12 May 2017

2017-05-12T11:00:00Z

(image)

Medical ImageNet, NVIDIA GTC, corporate responsibility in tech, online pricing

  1. Medical ImageNet — Drawing inspiration from the original ImageNet project led by Fei-Fei Li, Curt Langlotz’s lab at Stanford University has been building a Medical ImageNet repository that "contains 0.5 petabyte of clinical radiology data, comprising 4.5 million studies, and over 1 billion images.” Work is still underway, but they expect to release these data sets soon.
  2. Jensen Huang at GTC 2017 — NVIDIA’s GTC developer conference took place this week with founder and CEO Jensen Huang taking the stage to deliver the keynote. It took a full two hours, but Engadget has compiled a 13-minute highlight reel to fill you in. NVIDIA shareholders must be thrilled with Jensen’s announcements, as NVIDIA stock got a nice bump following his keynote.
  3. Satya Nadella fighting dystopia — Meanwhile at Microsoft’s own developer conference, BUILD, CEO Satya Nadella referenced George Orwell and Aldous Huxley to espouse technology companies demonstrating corporate and social responsibility.
  4. "How Online Shopping Makes Suckers of Us All" — "Our ability to know the price of anything, anytime, anywhere, has given us, the consumers, so much power that retailers—in a desperate effort to regain the upper hand, or at least avoid extinction—are now staring back through the screen. They are comparison shopping us.” Ouch. Data and machine learning have empowered online sellers to master pricing elasticity and consumer dollar extraction.

Continue reading Intelligent Bits: 12 May 2017.

(image)



Why self assessments improve learning

2017-05-12T10:00:00Z

O’Reilly’s assessment tool puts the focus on the learner, not arbitrary scores.We just don’t get it. Looking across the industry, most services promoted as “assessments” are basically memory tests or trivial click-to-own certifications. In one case, we discovered a site where 10 clicks through a video path was all that it took to get awarded a certificate, obtained in less than 10 seconds. Moreover, too much emphasis gets placed on scores provided to some third party, not on feedback to professionals so they can improve their learning experiences. We decided learners deserve better and we can deliver better. The Safari learning platform just made learning more effective by bringing self assessments to our popular Learning Paths. We’re adding more structure and support for those wanting to master a given subject, accelerating your “speed to understanding” by combining our expert curated Learning Path with new self assessments available exclusively in Safari. Different assessments for different outcomes While a variety of approaches exist, let’s consider the contrast between summative assessment and the formative assessment we use as a self assessment. Summative assessment measures how much students have learned up to a particular point in time, generally to meet some standard. Examples include final exams in university courses, or professional certifications. While those serve important needs in testing, their results are intended for someone other than the person taking the exam. Think: grades. In contrast, formative assessment gives feedback during testing. It’s considered part of the learning experience, and need not be graded. Questions are constructed such that if you understand the material, the answers are quick. On the other hand, if you’re struggling with a subject, you’ll need to spend much more time working through the questions. After carefully evaluating the options, O’Reilly chose to use formative assessment for the new self assessments in our Learning Paths. In other words, you get feedback for each answer. This approach is found in some state-of-the-art K-12 online learning, though rare in adult education and almost nonexistent for online learning in industry. When a professional is working through a self-paced Learning Path, having several waypoints of self assessment provides the feedback needed. It’s what is most appropriate for the medium. More to the point, it’s what’s most relevant for the learner. How assessments enhance learning Part of our[...]



How user stories propel Agile development

2017-05-12T10:00:00Z

(image)

User stories help track goals, user needs, and project timeframes to deliver valuable software quickly.

Continue reading How user stories propel Agile development.

(image)



Four short links: 12 May 2017

2017-05-12T09:00:00Z

Amazons Competes with Investment, Answering Questions, Designing for Survivors, and Open Source Support

  1. Amazon Chows Into Its Seed Corn -- Amazon invested in Nucleus via the Alexa Fund, then released their own version of Nucleus' functionality. The move will also likely deal a blow to the Alexa Fund, the investment vehicle through which Amazon has been supporting startups building products and services to be controlled by voice.
  2. Inferring and Executing Programs for Visual Reasoning​​ -- Facebook Research's paper that uses deep learning to answer questions like "Does the small sphere have the same color as the cube left of the gray cube?". Code released on github. (via @PyTorch)
  3. Privacy & Security Practices when Coping with Intimate Partner Abuse -- Google paper that combines technology practices with three phases of abuse to provide an empirically sound method for technology creators to consider how survivors of IPA can leverage new and existing technologies. Overall, our results suggest that the usability of and control over privacy and security functions should be or continue to be high priorities for technology creators seeking ways to better support survivors of IPA. (via Martin Shelton)
  4. How the TensorFlow Team Handles Open Source Support (Pete Warden) -- A successful open source project is a denial-of-service attack on its maintainers' time, so it's really interesting to see how the Google team both prioritised support and automated much of the drudgery around it.

Continue reading Four short links: 12 May 2017.

(image)



Open source and open standards in VR

2017-05-11T20:00:00Z

(image)

Stephanie Hurlburt explains why an open ecosystem is essential for the survival of virtual reality.

Continue reading Open source and open standards in VR.

(image)