Subscribe: O'Reilly Radar - Insight, analysis, and research about emerging technologies
http://radar.oreilly.com/atom.xml
Added By: Feedage Forager Feedage Grade B rated
Language: English
Tags:
continue reading  continue  data  design  learning  links march  machine learning  machine  open  reading  short links  world 
Rate this Feed
Rate this feedRate this feedRate this feedRate this feedRate this feed
Rate this feed 1 starRate this feed 2 starRate this feed 3 starRate this feed 4 starRate this feed 5 star

Comments (0)

Feed Details and Statistics Feed Statistics
Preview: O'Reilly Radar - Insight, analysis, and research about emerging technologies

All - O'Reilly Media



All of our Ideas and Learning material from all of our topics.



Updated: 2017-03-23T06:07:06Z

 



Working backwards

2017-03-22T20:00:00Z

(image)

Alan Cooper outlines his process for working backwards: Taking the time to ask the hard questions before wading into new territory.

Continue reading Working backwards.

(image)



Fireside chat with John Allspaw and Randy Hunt

2017-03-22T20:00:00Z

(image)

Etsy’s John Allspaw and Randy J. Hunt discuss the practices that have helped their tech and design teams evolve together.

Continue reading Fireside chat with John Allspaw and Randy Hunt.

(image)



Should designers. . .? New design skills from coding to Agile, process, and more

2017-03-22T20:00:00Z

(image)

Dan Mall shares perspectives on whether designers should code (yes!), how designers can fit into Agile workflows, and more.

Continue reading Should designers. . .? New design skills from coding to Agile, process, and more.

(image)



Government services that work for people

2017-03-22T20:00:00Z

(image)

Jennifer Pahlka says poor service design can have devastating consequences for vulnerable people in our country, but it doesn’t have to be that way.

Continue reading Government services that work for people.

(image)



From enterprise service bus to microservices

2017-03-22T11:00:00Z

The present and future of data integration in the cloud.In this episode of the O’Reilly podcast, Jon Bruner sat down with Rahul Kamdar, director of product management and strategy at TIBCO Software. They discussed the shift from the centralized enterprise service bus (ESB) to a distributed data architecture based on microservices, APIs, and cloud-native applications. src="https://w.soundcloud.com/player/?url=https%3A//api.soundcloud.com/tracks/313639516&auto_play=false&hide_related=false&show_artwork=true" height="166" width="100%" frameborder="no" scrolling="no"> Here are some highlights from their conversation: Cheaper, more scalable, and open to broader interaction In some ways, [microservices] are derived from the traditional service oriented architecture (SOA) style of services. But really, they represent the niche of that architecture, in terms of the set of practices that you would follow to build the microservices so they are easy to develop, and less expensive to manage, operate, and deploy. Microservices need to be elastic and scalable when taken to concepts such as the cloud. The second aspect is obviously around APIs—something that is key to companies pretty much all over the world right now—both from a technology standpoint as well as from a business standpoint. APIs extend the concept of microservices to make them available for external consumers in the form of the APIs, which is really defining the services in a standardized format and making them available in a consumable manner for developers and third-party partners alike. Changing your architecture means changing your culture For most users and developers who are moving toward this kind of architecture, I think the first thing that is really, really important is that it's critical to embrace or accept the fact that change is required not only on the technology side, but in many cases, on the organization or the culture side. Not only do you break down your applications from large, what are traditionally called monolithic applications, but, really you break down your teams into more functional groups that are potentially working in a very agile way—an agile scrum way in many cases—so they can reach their end results a lot more quickly, a lot more easily, and at the same time, focus on a very specific common functional set of applications they want to build. What’s coming down the road? In the new world, you're still going to build distributor applications and architecture that supports them, but at the same time, the set of different systems, the set of different data sources and applications that you still need to talk to, is going to remain, and probably get more complex because of hybrid deployments. Some customers who have always been on-premise or in a private datacenter are starting to deploy multiple clouds, or in some cases, something is on a private cloud, something is on a public cloud, and something is on third-party partners. They still want all their systems to be able to talk to each other. Integration by itself is starting to become even more key, but how it's being done is definitely changing. ESBs are mostly for the traditional architectures that still remain, but all the new ones are likely to adopt a more distributed integration architecture. This post is part of a collaboration between O’Reilly and TIBCO. See our statement of editorial independence. Continue reading From enterprise service bus to microservices.[...]



Four short links: 22 March 2017

2017-03-22T11:00:00Z

W3C Sucks, 2038 Ahoy, Swagger 3.0, and Javascript Bundling

  1. W3C Enabling Suing Researchers (BoingBoing) -- your periodic reminder that the W3C is captured by the enemies of open.
  2. 2038 Just 21 Years Away (LWN) -- an update on work to ease the 2038 problem. 2038 is my retirement insurance policy. [T]he point in early 2038 when 32-bit time_t values can no longer represent times correctly is now less than 21 years away. That may seem like a long time, but the relatively long life cycle of many embedded systems means that some systems deployed today will still be in service when that deadline hits.
  3. Visual Guide to What's New in Swagger 3.0 -- Swagger is a sweet way to define and document an API. I do like the side-by-side diffs showing old and new ways to do things, as a good way to communicate changes.
  4. Getting Started with Javascript Bundling and Webpack (YouTube) -- In this talk from nz(con), Tanya Grey teaches basic bundling, and this functions as a good walk through the mysterious world of Javascript tooling ... all those things you need to Do The Javascripts Goodly, those packages with names like "grunt," "gulp," and "browserify."

Continue reading Four short links: 22 March 2017.

(image)



Avoid testing headaches with tightly coupled services

2017-03-22T10:00:00Z

(image)

What you need to consider when your microservices architecture is tightly coupled.

Continue reading Avoid testing headaches with tightly coupled services.

(image)



Design for 7 billion. Design for one.

2017-03-21T20:00:00Z

(image)

Kat Holmes explores how designing for human diversity can unlock more meaningful experiences for each of us.

Continue reading Design for 7 billion. Design for one..

(image)



Highlights from the O'Reilly Design Conference in San Francisco 2017

2017-03-21T20:00:00Z

Watch highlights covering design thinking, UX, interaction design, and more. From the O'Reilly Design Conference in San Francisco 2017.Experts from across the design world came together in San Francisco for the O'Reilly Design Conference. Below you'll find links to highlights from the event. Design for 7 billion. Design for one. Kat Holmes explores how designing for human diversity can unlock more meaningful experiences for each of us. Watch "Design for 7 billion. Design for one." The expanding perimeter: The evolution of design in Silicon Valley Barry Katz explains how design has evolved from packaging electronics in sheet metal enclosures in the 1950s to grappling with some of the most fundamental problems of modern civilization. Watch "The expanding perimeter: The evolution of design in Silicon Valley." The UX of buildings, cities, and infrastructure Dan Hill explores creating great user experience for buildings and cities. Watch "The UX of buildings, cities, and infrastructure." Determining success in design Julie Zhuo discusses definitions of success in design and shares a few principles on how to set clear goals. Watch "Determining success in design." How is design driving outcomes? Doug Powell explains how IBM’s design transformation is shaping the company. Watch "How is design driving outcomes?" Fireside chat with Irene Au and Ivy Ross Khosla Ventures' Irene Au and Google's Ivy Ross discuss the future of hardware design, how to design products with soul, and lessons on design leadership. Watch "Fireside chat with Irene Au and Ivy Ross." Working backwards Alan Cooper outlines his process for working backwards: taking the time to ask the hard questions before wading into new territory. Watch "Working backwards." Should designers. . .? New design skills from coding to Agile, process, and more Dan Mall shares perspectives on whether designers should code (yes!), how designers can fit into Agile workflows, and more. Watch "Should designers. . .? New design skills from coding to Agile, process, and more." Government services that work for people Jennifer Pahlka says poor service design can have devastating consequences for vulnerable people in our country, but it doesn’t have to be that way. Watch "Government services that work for people." Fireside chat with John Allspaw and Randy Hunt Etsy’s John Allspaw and Randy J. Hunt discuss the practices that have helped their tech and design teams evolve together. Watch "Fireside chat with John Allspaw and Randy Hunt." Continue reading Highlights from the O'Reilly Design Conference in San Francisco 2017.[...]



Determining success in design

2017-03-21T20:00:00Z

(image)

Julie Zhuo discusses definitions of success in design and shares a few principles on how to set clear goals.

Continue reading Determining success in design.

(image)



How is design driving outcomes?

2017-03-21T20:00:00Z

(image)

Doug Powell explains how IBM’s design transformation is shaping the company.

Continue reading How is design driving outcomes?.

(image)



Fireside chat with Irene Au and Ivy Ross

2017-03-21T20:00:00Z

(image)

Khosla Ventures' Irene Au and Google's Ivy Ross discuss the future of hardware design, how to design products with soul, and lessons on design leadership.

Continue reading Fireside chat with Irene Au and Ivy Ross.

(image)



The expanding perimeter: The evolution of design in Silicon Valley

2017-03-21T20:00:00Z

(image)

Barry Katz explains how design has evolved from packaging electronics in sheet metal enclosures in the 1950s to grappling with some of the most fundamental problems of modern civilization.

Continue reading The expanding perimeter: The evolution of design in Silicon Valley.

(image)



The UX of buildings, cities, and infrastructure

2017-03-21T20:00:00Z

(image)

Dan Hill explores creating great user experience for buildings and cities.

Continue reading The UX of buildings, cities, and infrastructure.

(image)



Four short links: 21 March 2017

2017-03-21T10:55:00Z

Face Scanners, Formal Specifications, Simulated NYC, and Open Source Motorbike

  1. Wiping Out Crime -- face scanners in Beijing public toilets to ration out toilet paper.
  2. Video Course in TLA+ -- Leslie Lamport's course on his specification language.
  3. Humans of Simulated New York -- somewhere in the simulation, a data structure skims Four Short Links and thinks "that's just silly." The model presented in this paper experiments with a comprehensive simulant agent in order to provide an exploratory platform in which simulation modelers may try alternative scenarios and participation in policy decision-making.
  4. Open Source Motorcycle -- putting the forks back in ... no, I can't just do it. I'm sorry.

Continue reading Four short links: 21 March 2017.

(image)



How can I find out where all my disk space has gone on a Linux file system?

2017-03-21T08:00:00Z

(image)

Learn how to use the Linux du command to discover what directories are consuming the most space in your file system.

Continue reading How can I find out where all my disk space has gone on a Linux file system?.

(image)



How can I find duplicate files in Linux?

2017-03-21T08:00:00Z

(image)

Learn how to identify duplicate copies of files in your Linux system allowing you to be more organized and save disk space.

Continue reading How can I find duplicate files in Linux?.

(image)



How can I make the Linux spell command more useful?

2017-03-21T08:00:00Z

(image)

Learn how to deal with the shortcomings of the Linux spell command; such as unsorted output, duplicates, and false positives.

Continue reading How can I make the Linux spell command more useful?.

(image)



Four short links: 20 March 2017

2017-03-20T11:10:00Z

Time Series Database, Open Source Maintenance, Conversational Devices, and Google Glass's Act Two

  1. TimescaleDB -- an open source time series database optimized for fast ingest and complex queries. Fully compatible with Postgres.
  2. Managing an Open Source Project (Daniel Bachhuber) -- always interesting to see how open source maintainers manage their time and the flow of demands on it.
  3. Tom Coates on Conversational Devices -- podcast with Mr. Coates from Thington, talking about seamfulness, semantics, and the complexity of the scenarios of your home. If you disaggregate what people say they want, it gets more complicated—which leads into a thoughtful and restrained discussion of why "I just want the lights to come on when I move in my bedroom" might not be welcome if implemented in a straightforward fashion.
  4. Google Glass's Second Life in Manufacturing -- With Google Glass, she scans the serial number on the part she's working on. This brings up manuals, photos, or videos she may need. She can tap the side of headset or say "OK Glass" and use voice commands to leave notes for the next shift worker. Because your shift manager doesn't call you a "Glasshole" if you use it to do your job.

Continue reading Four short links: 20 March 2017.

(image)



2017 Design Salary Survey

2017-03-20T11:10:00Z

(image)

Tools, trends, titles, what pays (and what doesn’t) for design professionals

atlas book skeleton

Executive Summary

THE 2017 O’REILLY DESIGN SALARY SURVEY explores the landscape of modern design professionals, giving details about their roles and how much they earn. The results are based on data from our online survey that collected 1,085 responses. We pay special attention to variables that correlate with salary, but this report isn’t just about money—we present a range of information, including the popularity of design tools, tasks, and organizational processes.

Continue reading 2017 Design Salary Survey.

(image)



On computational ethics

2017-03-20T11:00:00Z

Is it possible to imagine an AI that can compute ethics?While I was writing my post about artificial intelligence and aggression, an odd thought occurred to me. When we discuss the ethics of artificial intelligence, we're typically discussing the act of creating (programming, training, and deploying) AI systems. But there's another way to look at this. What kind of ethics is amenable to computation? Can ethical decision-making be computable? I wasn't quite serious when I wrote about AIs deriving Asimov's Laws of Robotics from first principles—I really doubt that's possible. In particular, I doubt that AIs will ever be able to "learn" the difference between humans and other things: dogs, rocks, or, for that matter, computer-generated images of humans. There's a big difference between shooting a human and shooting an image of a human in Overwatch. How is a poor AI, which only sees patterns of bits, supposed to tell the difference? But for the sake of argument and fantasy, let's engage in a thought experiment where we assume that AIs can tell the difference. And that the AI has unlimited computational power and memory. After all, Asimov was writing about precisely predicting human actions centuries in the future based on data and statistics. Big data indeed—and any computers he knew were infinitesimally small compared to ours. So, with computers that have more computational power than we can imagine today: is it possible to imagine an AI that can compute ethics? Jeremy Bentham summarized his moral philosophy, which came to be known as utilitarianism, by saying "the greatest happiness of the greatest number is the foundation of morals and legislation." That sounds like an optimization problem: figure out a utility function, some sort of computational version of Maslow's hierarchy of needs. Apply it to every person in the world, figure out how to combine the results, and optimize: use gradient ascent to find the maximum total utility. Many problems would need solutions along the way: what is that utility function in the first place, how do you know you have the right one, is the utility function the same for every person, and how do you combine the results? Is it simply additive? And there are problems with utilitarianism itself. It's been used to justify eugenics, ethnic cleansing, and all sorts of abominations: "the world would really be a better place without ..." Formally, this could be viewed simply as choosing the wrong utility function. But the problem isn't that simple: how do you know you have the right utility function, particularly if it can vary from person to person? So, utilitarianism might, in theory, be computable, but choosing appropriate utility functions would be subject to all sorts of prejudices and biases. Immanual Kant's ethics might also be computable. The primary idea, "act according to the principle which you would want to be a universal law" (the "categorical imperative"), can also be rephrased as an optimization problem. If I want to eat ice cream every day, I should first think about whether the world would be a tolerable place if everyone ate ice cream every day. It probably would (though that might conceivably push us up against some boundary conditions in the milk supply). If I want to dump my industrial waste on my neighbors' lawns, I have to think about whether the world would be acceptable if everyone did that. It probably wouldn't be. Computationally, this isn't all that different from utilitarianism: I have to apply my rule to everyone, su[...]



Four short links: 17 March 2017

2017-03-17T11:30:00Z

Personalized Learning, Programming Programming Languages, Poker AI, and Technical Interviews

  1. Problems with Personalized Learning (Dan Meyer) -- a thorough and beautiful skewering of vapid edtech promises. Personalized learning is only as good as its technology, and in 2017, that technology isn’t good enough.
  2. Beautiful Racket -- a book on making programming languages, one that's written to be readable instead of academic.
  3. DeepStack: Expert-Level Artificial Intelligence in Heads-Up No-Limit Poker -- now the AIs are kicking ass at heads-up no-limit Texas Hold 'em.
  4. Acing the Technical Interview (Aphyr) -- satire, I hope. Beautiful beautiful satire.

Continue reading Four short links: 17 March 2017.

(image)



How does Facebook recognize my face and the faces of friends and family?

2017-03-17T08:00:00Z

(image)

Learn how Facebook and other trailblazers use AI technologies to recognize human features.

Continue reading How does Facebook recognize my face and the faces of friends and family?.

(image)



How does IBM Watson search TED Talks?

2017-03-17T08:00:00Z

(image)

Learn how you can open up non-text content to search with deep learning.

Continue reading How does IBM Watson search TED Talks?.

(image)



How does Google's driverless car work?

2017-03-17T08:00:00Z

(image)

Learn how deep learning has accelerated the realization of driverless vehicles and what that means for the future.

Continue reading How does Google's driverless car work?.

(image)



Driving enterprise open source adoption, from data lake to AI

2017-03-16T20:00:00Z

(image)

Ron Bodkin explains how Teradata encourages open source adoption within enterprises.

Continue reading Driving enterprise open source adoption, from data lake to AI.

(image)



Machine learning at Google

2017-03-16T20:00:00Z

(image)

Rob Craft shares some of the ways machine learning is being used inside of Google.

Continue reading Machine learning at Google.

(image)



Data in disasters: Saving lives and innovating in real time

2017-03-16T20:00:00Z

(image)

Desi Matel-Anderson dives into the world of the Field Innovation Team, which uses data to save lives during disasters.

Continue reading Data in disasters: Saving lives and innovating in real time.

(image)



Big data, AI, the genome, and everything

2017-03-16T20:00:00Z

(image)

Vijay Narayanan explains how cloud, data, and artificial intelligence are accelerating the genomic revolution.

Continue reading Big data, AI, the genome, and everything .

(image)



Becoming smarter about credible news

2017-03-16T20:00:00Z

(image)

Cloudera CEO Tom Riley and Thomson Reuters VP of R&D Khalid Al-Kofahi discuss big data's role in chasing down leads, verifying sources, and determining what's newsworthy.

Continue reading Becoming smarter about credible news.

(image)



Making good robots

2017-03-16T20:00:00Z

(image)

Andra Keay outlines principles of good robot design and discusses the implications of implicit bias in our robots.

Continue reading Making good robots.

(image)



Machine learning is about your data and deployment, not just model development

2017-03-16T20:00:00Z

(image)

Dinesh Nirmal discusses how your data can help you build the right cognitive systems to engage with your customers.

Continue reading Machine learning is about your data and deployment, not just model development .

(image)



Ray: A distributed execution framework for emerging AI applications

2017-03-16T20:00:00Z

(image)

Michael I. Jordan explores applications in artificial intelligence.

Continue reading Ray: A distributed execution framework for emerging AI applications.

(image)



Noah Iliinsky on design at Amazon

2017-03-16T11:55:00Z

(image)

The O’Reilly Design Podcast: The importance of intentional thinking, user-centered data visualizations, and separating functionality from implementation.

In this week’s Design Podcast, I sit down with Noah Iliinsky, senior UX architect at Amazon’s AWS group, co-author of Designing Data Visualizations, and co-editor of Beautiful Visualization. We talk about how design is organized at Amazon, 17 keys to success, and why being intentional will ensure you are working on the right problems.

Continue reading Noah Iliinsky on design at Amazon.

(image)



Four short links: 16 March 2017

2017-03-16T11:25:00Z

Werewolf AI, Board Games, Coin Tossing, and Glitch Platform

  1. Towards Deception Detection in a Language-Driven Game (PDF) -- This paper focuses exclusively on how the Explanation Generator generates hypotheses for the actions of human players based on observations of their conversational utterances. Werewolf is their test data. I do not think it is wise to teach the softwares to play Werewolf.
  2. CIA Trains Officers with Board Games (Ars Technica) -- where are the software/startup simulation board games? (via BoingBoing)
  3. The Impact of a Coin Toss on Major Life Decisions and Subsequent Happiness (PDF) -- Those who flipped heads were approximately 25% more likely to report making a change than those who got tails.
  4. Glitch -- sweet collaboratively edited code for web apps, with View Source, but clearly laying a path to being commercial PaaS. Neat.

Continue reading Four short links: 16 March 2017.

(image)



Build a talking, face-recognizing doorbell for about $100

2017-03-16T11:00:00Z

DIY with Amazon Echo and Raspberry PI: Recognize thousands of people at your door every month, for pennies.Recently, I set out to install a doorbell in my new house and thought: why doesn’t my doorbell tell me who is at the door? Most of my DIY projects end up costing more than the equivalent product, even if I value my time at $0 per hour. Something about supply chains and economies of scale, I guess. (But I have way more fun making things myself.) In the course of this project, I built a door camera that is not only way cheaper than my Dropcam, but it has some genuinely useful features that, for some reason, aren’t available on the market yet. Figure 1. My front door with a doorbell, August Lock keypad and face-recognizing Raspberry Pi. Image by Lukas Biewald. Here’s what we’re going to build: a $60 Raspberry Pi-powered security camera setup that takes pictures, posts them to the cloud, and then does face recognition. You could also stream the data to Amazon S3, making it a full-fledged Dropcam replacement. While Nest charges $100 a year for keeping the last 10 days of video footage, you could keep a year of camera footage in S3 for around $20. If you used Amazon Glacier, that cost would go down to around $4. Machine learning with Amazon Rekognition This tutorial will focus on the machine learning part—using Amazon’s new Rekognition service to do face recognition on your guests, and send that to your Amazon Echo so you will always know who’s at your door. In order to build a reliable service, we’ll also make use of one of Amazon’s coolest and most useful products: Lambda. Ingredients: Amazon Echo Dot ($50) Raspberry PI V3 ($38) (This project would also work with a Pi v2 and USB Wifi) Raspberry PI-Compatible Camera ($16) Raspberry Pi Case ($6) 16GB SD Card ($8) Total: $118 We will use Amazon’s S3, Lambda, and Rekognition services for the face matching. These services are free to get started, and you can recognize thousands of people at your door every month for pennies. Setting up the Raspberry Pi If you’ve done any of my other Raspberry Pi tutorials, much of this will be familiar. First, download Noobs from the Raspberry Pi Foundation and follow their setup instructions. This mainly involves copying Noobs onto an SD card and then plugging the SD card into your Pi. Then plug a mouse, keyboard, and monitor into your Pi and follow the setup instructions, which have gotten much more accessible since the launch of Pixel, the new desktop environment. Figure 2. Raspberry Pi on my desk with tiny monitor and keyboard. Image by Lukas Biewald. Next, change the name of your Pi to something you can remember, so you can SSH into it. There’s good instructions for this on howtogeek—you need to modify the /etc/hosts and /etc/hostname files and give your Pi a name. I like to name all of my security camera Raspberry Pi after characters on my favorite TV show, “It’s Always Sunny in Philadelphia,” so I named my front door camera “Dennis.” That means I don’t need to remember an IP address, and I c[...]



5 tips for embracing open data science in the enterprise

2017-03-16T11:00:00Z

Transform the way you approach analytics.Businesses are continually seeking competitive advantage. Lately, the focus has been on leveraging data to seize opportunities, detect possible weaknesses, and triumph over competitors. Big data, in particular, offers a multitude of ways to use data to drive strategic, operational, and execution practices. And, increasingly, data science is the way to get there. First, a definition: data science is a multidisciplinary field that combines the latest innovations in advanced analytics—including machine learning and artificial intelligence—with high-performance computing and visualizations to extract knowledge or insights from data. The tools of data science originated in the scientific community, where researchers used them to test and verify hypotheses that include “unknown unknowns.” These tools have entered business, government, and other organizations gradually over the past 10 years as computing costs have dropped and software has grown more sophisticated. But proprietary tools and technologies have proved to be inadequate to support the speed and innovation happening in the data science world. Enter the open source community. Open source communities want to break free from the shackles of proprietary tools and embrace a more open and collaborative work style that reflects the way they work—with teams distributed all over the world. These communities are not just creating new tools; they’re calling on enterprises to use the right tools for the problems at hand. Open data science is revolutionary. It transforms the way organizations approach analytics. With open data science, you can boost the productivity of your data team, enhance efficiencies by moving to a self-service data model, and overcome organizational and technical barriers to making the most of your big data. Here are five things you can do to embrace open data science: Wholeheartedly adopt open source. Traditional commercial data science tools evolve slowly. Although stable and predictable, many of them have been architected around 1980s-style client-server models that don’t scale to internet-oriented deployments with web-accessible interfaces. On the other hand, the open data science ecosystem is founded on concepts of standards, openness, web accessibility, and web-scale-oriented distributed computing. And, open data science tools are created by a global community of analysts, engineers, statisticians, and computer scientists who have hands-on experience in the field. This global community includes millions of users and developers who rapidly iterate the design and implementation of the most exciting algorithms, visualization strategies, and data processing routines available today. These pieces can be scaled and deployed efficiently and economically to a wide range of systems. By enthusiastically adopting—and contributing to—this community, your chances of having successful deployments multiplies exponentially. Build a data science team with diverse skills. Success[...]



The machine learning renaissance

2017-03-15T20:00:00Z

(image)

Mike Olson says without big data and a platform to manage big data, machine learning and artificial intelligence just don’t work.

Continue reading The machine learning renaissance.

(image)



Highlights from Strata + Hadoop World in San Jose 2017

2017-03-15T20:00:00Z

(image)

Watch highlights covering data science, data engineering, data-driven business, and more. From Strata + Hadoop World in San Jose 2017.

Experts from across the data world are coming together in San Jose for Strata + Hadoop world. Below you'll find links to highlights from the event.

Continue reading Highlights from Strata + Hadoop World in San Jose 2017.

(image)



Applying data and machine learning to scale education

2017-03-15T20:00:00Z

(image)

Daphne Koller explains how Coursera is using large-scale data processing and machine learning in online education.

Continue reading Applying data and machine learning to scale education.

(image)



Turning the internet upside down: Driving big data right to the edge

2017-03-15T20:00:00Z

(image)

Ted Dunning says the internet of things is turning the internet upside down, and the effects are causing all kinds of problems.

Continue reading Turning the internet upside down: Driving big data right to the edge.

(image)



Machines and the magic of fast learning

2017-03-15T20:00:00Z

(image)

Eric Frenkiel looks at advanced tools and use cases that demonstrate the power of machine learning to bring about positive change.

Continue reading Machines and the magic of fast learning .

(image)



Collaboration in AI benefits humanity

2017-03-15T20:00:00Z

(image)

Jason Waxman says collaboration between industry, government, and academia is needed to deliver on the promise of AI.

Continue reading Collaboration in AI benefits humanity.

(image)



Launching Pokémon GO

2017-03-15T20:00:00Z

(image)

Phil Keslin, CTO of Niantic, explains how the engineering team prepared for—and just barely survived—the experience of launching Pokémon GO.

Continue reading Launching Pokémon GO.

(image)



A vision for future cybersecurity

2017-03-15T12:00:00Z

(image)

Consolidating cybersecurity for a more secure future.

Continue reading A vision for future cybersecurity.

(image)



Scout Brody on crafting usable and secure technologies

2017-03-15T11:19:00Z

(image)

The O’Reilly Security Podcast: Building systems that help humans, designing better tools through user studies, and balancing the demands of shipping software with security.

In this episode, O’Reilly Media’s Mac Slocum talks with Scout Brody, executive director of Simply Secure. They discuss building systems that help humans, designing better tools through user studies, and balancing the demands of shipping software with security.

Continue reading Scout Brody on crafting usable and secure technologies.

(image)



The designer's perspective on prototyping

2017-03-15T11:15:00Z

(image)

Why it’s important for designers to make ideas tangible and testable.

The Explosion of Prototyping Tools

The gap in design tooling seems obvious in retrospect. As the world goes digital—as software consumes it, to paraphrase Marc Andreessen—it only makes sense that software designers need digital toolsets to facilitate their work and craft. When we first began this journey to digitize the world, designers naturally found inspiration in and imported best practices from other disciplines: architecture, industrial and print design, ethnography, and computer science, among many others.

It would be wrong to say the early years of the web and mobile that design tooling did not exist: we had applications that helped us do the work, but they weren’t always exactly the right fit. Unsurprisingly, the first websites and web apps were prototyped using tools repurposed for the new digital medium.

Continue reading The designer's perspective on prototyping.

(image)



Ideas on interpreting machine learning

2017-03-15T11:15:00Z

Mix-and-match approaches for visualizing data and interpreting machine learning models and results.You’ve probably heard by now that machine learning algorithms can use big data to predict whether a donor will give to a charity, whether an infant in a NICU will develop sepsis, whether a customer will respond to an ad, and on and on. Machine learning can even drive cars and predict elections. ... Err, wait. Can it? I believe it can, but these recent high-profile hiccups should leave everyone who works with data (big or not) and machine learning algorithms asking themselves some very hard questions: do I understand my data? Do I understand the model and answers my machine learning algorithm is giving me? And do I trust these answers? Unfortunately, the complexity that bestows the extraordinary predictive abilities on machine learning algorithms also makes the answers the algorithms produce hard to understand, and maybe even hard to trust. Although it is possible to enforce monotonicity constraints (a relationship that only changes in one direction) between independent variables and a machine-learned response function, machine learning algorithms tend to create nonlinear, non-monotonic, non-polynomial, and even non-continuous functions that approximate the relationship between independent and dependent variables in a data set. (This relationship might also be referred to as the conditional distribution of the dependent variables, given the values of the independent variables.) These functions can then make very specific predictions about the values of dependent variables for new data—whether a donor will give to a charity, an infant in a NICU will develop sepsis, a customer will respond to an ad, etc. Conversely, traditional linear models tend to create linear, monotonic, and continuous functions to approximate the very same relationships. Even though they’re not always the most accurate predictors, the elegant simplicity of linear models makes the results they generate easy to interpret. While understanding and trusting models and results is a general requirement for good (data) science, model interpretability is a serious legal mandate in the regulated verticals of banking, insurance, and other industries. Business analysts, doctors, and industry researchers simply must understand and trust their models and modeling results. For this reason, linear models were the go-to applied predictive modeling tool for decades, even though it usually meant giving up a couple points on the accuracy scale. Today, many organizations and individuals are embracing machine learning algorithms for predictive modeling tasks, but difficulties in interpretation still present a barrier for the widespread, practical use of machine learning algorithms. In this article, I pr[...]



Four short links: 15 Mar 2017

2017-03-15T11:00:00Z

VR/AR Harm, Vulnerable Containers, Old-School Coding, and Complexity + Strategy

  1. Still Logged In: What AR and VR Can Learn from MMOs -- Raph Koster's GDC keynote forcefully makes the point that online immersive experiences are disproportionately used by people who are emotionally vulnerable, yet VR/AR is recreating the tragic mistakes made by game designers. (via BoingBoing)
  2. Docker Image Vulnerability Research -- 24% of the latest Docker images have significant vulnerabilities.
  3. 1965 Intro to Programming Course (PDF) -- old-school flowcharts to code, but I couldn't find a paragraph to quote because it's full of "the student ... his ... he ... him," which waters eyes these days.
  4. Complexity and Strategy -- In actual practice, if the product stays small, you can essentially “book” that initial productivity gain—a clear win. If the product starts to grow complex—and you can predict that fairly directly by looking at the size of the development team—then costs will come to be dominated by that increasing feature interaction and essential complexity. Project after project has demonstrated there is nothing about language or underlying technical infrastructure that changes that fundamental curve.

Continue reading Four short links: 15 Mar 2017.

(image)



Drawing a map of distributed data systems

2017-03-15T11:00:00Z

How we created an illustrated guide to help you find your way through the data landscape. Designing Data-Intensive Applications, the book I’ve been working on for four years, is finally finished, and should be available in your favorite bookstore in the next week or two. An incomplete beta (Early Release) edition has been available for the last 2 1/2 years as I continued working on the final chapters. Throughout that process, we have been quietly working on a surprise. Something that has not been part of any of the Early Releases of the book. In fact, something that I have never seen in any tech book. And today we are excited to share it with you. In Designing Data-Intensive Applications, each of the 12 chapters is accompanied by a map. The map is a kind of graphical table of contents of the chapter, showing how some of the main ideas in the chapter relate to each other. Here is an example, from Chapter 3 (on storage engines): Figure 1. Map illustration from Designing Data-Intensive Applications, O’Reilly Media, 2017. Click here for a larger version. Don’t take it too seriously—some of it is a little tongue-in-cheek, we have taken some artistic license, and the things included on the map are not exhaustive. But it does reflect the structure of the chapter: political or geographic regions represent ways of doing something, and cities represent particular implementations of those approaches. Similar things are more likely to be close together, and roads or rivers represent concepts that connect different implementations or regions. Most computing books describe one particular piece of software and discuss all the aspects of how it works. This book is structured differently: it starts with the concepts—discussing the high-level approaches of how you might solve some problem, and comparing the pros and cons of each—and then points out which pieces of software use which approach. The maps use the same structure: the region in which a city is located tells you what approach it uses. For example, in the map above, you can see a high-level subdivision into two countries: transaction processing and analytics. Within transaction processing, there are two regions: log-structured storage and B-trees, which are two ways of implementing OLTP storage engines. Within the B-tree region, you see databases like MySQL and PostgreSQL[1], while within the log-structured region you see databases like Cassandra and HBase. On the analytics side, you can see that the mountain range representing column storage reaches into both the data warehousing and the Hadoop regions, since the approach applies to both. The maps are in black and white, both because of practicalities of printing and also b[...]



Instagram's 5 principles for implementing and scaling continuous deployment

2017-03-15T10:00:00Z

(image)

How to approach continuous deployment at your own organization.

Continue reading Instagram's 5 principles for implementing and scaling continuous deployment.

(image)



Saving the world—or at least the world’s scientific and government data

2017-03-14T13:50:00Z

(image)

The O’Reilly Data Show Podcast: Max Ogden on data preservation, distributed trust, and bringing cutting-edge technology to journalism.

In this special episode of the Data Show, O'Reilly's Jenn Webb speaks with Maxwell Ogden, director of Code for Science and Society. Recently, Ogden and Code for Science have been working on the ongoing rescue of data.gov and assisting with other data rescue projects, such as Data Refuge; they’re also the nonprofit developers supporting Dat, a data versioning and distribution manager, which came out of Ogden's work making government and scientific data open and accessible.

Continue reading Saving the world—or at least the world’s scientific and government data.

(image)



Business transformation starts with leadership transformation

2017-03-14T11:00:00Z

Set a good example, and everyone will follow.Leaders today are inundated with reasons to transform their organizations in search of better outcomes. New market entrants erode profit, competitors seem to be always moving ahead, all while customers seek higher quality and cheaper sources of service. The pace and appetite for change is exhausting. Yet comparatively, it feels like your organization is sinking deeper into the mud. “We want to change, but the culture here is too difficult to change.” It’s a frequent remark we have all heard and said, but what does it mean? Culture is the original business meme. Its meaning and usage are as abstract and intangible as the word itself. “If we just fix the culture, we will be successful”—a statement full of positive intent yet lacking a clear directive or step to take. A new culture is not a browser plugin. Leaders cannot simply select an extension, and download and install it from the web. Nor should leaders expect the update to be applied only to others and not to themselves. The prevailing thinking is the need to change people’s mindsets. The belief being if we tell people to think differently, they will act differently. All-hands meetings are called, PowerPoint decks are prepped, and an executive tour is scheduled to rally the troops for the mission ahead. A one-, maybe two-day training session is delivered, and the metamorphosis begins. But it does not. Culture is our behaviors. It is the actions we perform. The way we talk and how we treat one another. The way we behave reflects the values and expectations we have of ourselves and of one another. The single most important action of any leader is to role model the behaviors they wish to see others exhibit in the organization. Actions are what matter, not talk Culture change does not lead with words; it leads with action. By changing the way we behave, our actions begin to change the way we observe, experience, and eventually see the world. By seeing and experiencing the world differently, it changes the way we think about the world. People do not change their mental models of the world by speaking about it; they need to experience the change to believe and feel it. Figure 1. John Shook’s Change Model. Image re-creation by Barry O'Reilly. John Shook was the first American manager to be hired by Toyota. He moved to Japan without knowing a word of Japanese, just with a desire to immerse himself in the organization for a prolonged period of time to learn the Toyota Production System by doing it. What he observed was not a group of managers telling people what to think or how to perform[...]



Four short links: 14 March 2017

2017-03-14T10:55:00Z

Maps for Cars, Container Metrics, Game Patent, and QR Scams

  1. The Most Detailed Maps of the World Will Be for Cars Not Humans (Ars Technica) -- a great point, well stated.
  2. ctop -- top for container metrics.
  3. The Tapper Videogame Patent -- Video game in which a host image repels ravenous images by serving filled vessels.
  4. QR Code Scams -- paste your own QR code over the merchant's, and customers happily pay your account instead—e.g., Users normally can scan a code to unlock rental bikes; by attaching their own QR code to the bike, fraudsters can fool bike riders into transferring $43—the same amount as Mobike’s required deposit—to their account.

Continue reading Four short links: 14 March 2017.

(image)



What developers are doing (and not doing) to help with performance

2017-03-14T10:00:00Z

(image)

Poor load times hurt your UX and your bottom line.

Continue reading What developers are doing (and not doing) to help with performance.

(image)



Know-nothing authentication

2017-03-14T04:00:00Z

If behavioral authentication could be made to work, it could be a big part of our future.I was intrigued by an article about behavioral authentication on the Fast Forward Labs blog. Behavioral authentication is a kind of biometric authentication based on aspects of your behavior: timings while typing, for example, but conceivably much more fine-grained, like whether your fingers are centered on the touchscreen's virtual keys when you press them, how hard you press, how you perform multi-fingered gestures, and much more. There's a compelling argument against most forms of biometric authentication. Consider fingerprints. They're public; you leave them behind on glasses, they can be captured through photography that's within the capabilities of a good mobile phone, and they can't be changed if they're compromised. And if the bad guys have you, they can force you to scan your fingerprint (or just remove a few fingers). It's more difficult to steal a retinal scan, but my opthalmologist has mine, and who knows whether her office has good security practices? (There's a James Bond movie in which someone gets his retinas modified so he can get into Very Secret Places.) And voice: it's a trivial exercise to get a recording of just about anyone you're interested in, though voice recognition could be part of a behavioral authentication system. Behavioral authentication is different, in some odd but interesting ways. Two properties push it much further than anything I can imagine with other authentication mechanisms. First, it can be continuous, as the Fast Forward article points out. It's not a matter of entering a password or scanning a fingerprint that lets you in. You're interacting as long as you're using the device, and the authentication can (and should) continuously be authenticating. Second, you don't really know what it's using to authenticate you. Is it the force with which you press? It is timing? Is it something else? Could it be some combination of factors? Could the authentication factors (and their weights) be constantly shifting and changing? We talk about authentication tokens in terms of "something you know, something you have, or something you are." Passwords you clearly know. They're easily forgotten, and surprisingly easy to discover through a variety of attacks. Dongles and other security devices are things that you have; they're easily lost or stolen. Fingerprints are clearly things that you have, and as I've pointed out, they can also be captured. But behavioral patterns? I don't know how I type the way I do. My behaviors aren'[...]



Want to scale design thinking?

2017-03-13T21:05:00Z

8 tips for embracing digital design collaboration.Design is a top priority for companies that want to innovate and continue to improve the customer experiences. While many are investing in design thinking education, the distributed nature of teams makes it difficult for teams to practice, especially together. Ultimately, big initiatives to roll out design thinking as a core competence for all “knowledge and imagination workers” have seen friction for both newly minted design thinkers and experts. Onboarding education formats for design thinking typically rely heavily on accelerated in-person workshops where attendees leave with questions like: “How do I do this with my team that is spread all over the world?” Without a common place to practice what they’ve just learned, distributed teams are forced to either relocate or travel to put the methods to work. We see these common issues as we work with design and innovation leaders: After coming together for an in-person workshop, teams experience Workshop Amnesia, as their memory of the training fades while they wait for transcriptions and actionable items. A lack of dedicated project rooms makes it difficult for teams to continuously collaborate throughout a project. A lot of time is wasted aligning which methods should be used in a training session because of varied education levels and a lack of standardization. Rookies need to learn the methods as they go. How does an enterprise nurture a design culture when their teams are distributed...everywhere? Digital workplaces for design collaboration The digital workplace transformation has helped bring the global workforce in companies closer together. Video- and chat-based communications have rolled out virally, and teams use them daily for their work. It’s now time for design and innovation workshops, and sprints, to also go digital. Figure 1. MURAL on different devices. Modern teams at global enterprises like Accenture, IBM, Intuit, and Autodesk are already using digital tools and reaping the benefits. Courtesy of MURAL. These eight best practices will help you implement digital design collaboration for your teams: 1. Expand design studios and innovation centers—virtually. Companies have invested millions in well-equipped innovation centers designed specifically to foster creativity. By incorporating technology, you can remove the physical limits of a room or location and enable innovation to take place everywhere, at any time. Complement in-person workshops with[...]



Four short links: 13 March 2017

2017-03-13T10:45:00Z

Attention Prioritization, Event Sourcing, Containerized Dropbox, and Player Modeling

  1. ASAP: Automatic Smoothing for Attention Prioritization in Time Series -- automatically smooths time series plots to remove short-term noise while retaining large-scale deviations.
  2. PumpkinDB -- Event sourcing database engine that doesn't overwrite data.
  3. Run Dropbox in a Container -- keep its grubby fingers off your account.
  4. Ethical Considerations in Player Modeling -- We source categories of ethical issues in the application of artificial intelligence (AI) from work on AI ethics, and using these, we provide several specific examples of ethical issues in player modeling. Building from the examples, we suggest establishing a framework for understanding ethical issues in player modeling, and we propose a number of methodological approaches to address the identified challenges.

Continue reading Four short links: 13 March 2017.

(image)



Four short links: 10 March 2017

2017-03-10T12:05:00Z

Puma Surveillance, Illicit Domains, Ethics for Algorithms, and the Drama Triangle Puma Surveillance State Proceeds Apace (PDF) -- Acquiring reliable data on large felid populations is crucial for effective conservation and management. However, large felids, typically solitary, elusive, and nocturnal, are difficult to survey. [...] Classification accuracy was consistently > 90% for individuals, and for the correct classification of footprints within trails, and > 99% for sex classification. The technique has the potential to greatly augment the methods available for studying puma and other elusive felids, and is amenable to both citizen-science and opportunistic/local community data collection efforts, particularly as the data collection protocol is inexpensive and intuitive. I wonder whether dong deduction from footprint photos features in puma dystopic literature. Information Extraction in Illicit Domains (PDF) -- Illicit domains pose some formidable challenges for traditional IE systems, including deliberate information obfuscation, non-random misspellings of common words, high occurrences of out-of-vocabulary and uncommon words, frequent (and non-random) use of Unicode characters, sparse content and heterogeneous website structure, to only name a few. [...] We present a lightweight feature-agnostic information extraction system for a highly heterogeneous, illicit domain like human trafficking. Ethics for Powerful Algorithms (Abe Gong) -- video of Abe's talk at ODSC. He suggests four questions we should ask ourselves as we automate humans out of a loop: 1. Are the statistics solid? 2. Who wins? Who loses? 3. Are the changes in power structures helping? 4. How can we mitigate harms? (via O'Reilly) Karpman Drama Triangle -- I collect useful mental frameworks and models. This one does a great job of explaining "drama" (vs. genuine victimization), which you'll now recognize in interpersonal conflict at work and at home. The standard solution is the Winner's Triangle (where we should be vulnerable, caring, and assertive), but a book called The Power of TED suggests the participants look for roles as Creator, Challenger, and Coach for getting to a desired outcome. "You don't have to be a therapist to manage people, but it helps." Continue reading Four short links: 10 March 2017.[...]



How to use pull requests to improve your code reviews

2017-03-10T11:00:00Z

(image)

Spend more time building and less time fixing with GitHub Pull Requests for proper code review.

If you don’t write code every day, you may not know some of the problems that software developers face on a daily basis:

  • security vulnerabilities in the code
  • code that causes your application to crash
  • code that can be referred to as “technical debt” and needs to be re-written later
  • code that has already been written somewhere that you didn’t know about

Code review helps improve the software we write by allowing other people and/or tools to look it over for us. This review can happen with automated code analysis or test coverage tools — two important pieces of the software development process that can save hours of manual work — or peer review. Peer review is a process where one person manually reviews the code written by another person. When it comes to developing software, speed and urgency are two components that often result in some of the previously mentioned problems. If you don’t release soon enough, your competitor may come out with a new feature first. If you don’t release often enough, your users may doubt whether or not you still care about improvements to your application.

Continue reading How to use pull requests to improve your code reviews.

(image)