Subscribe: All Things Distributed
Added By: Feedage Forager Feedage Grade B rated
Language: English
amazon  applications  aws  business  cloud  companies  customers  data  digital  hong kong  new  systems  time  weekend   
Rate this Feed
Rate this feedRate this feedRate this feedRate this feedRate this feed
Rate this feed 1 starRate this feed 2 starRate this feed 3 starRate this feed 4 starRate this feed 5 star

Comments (0)

Feed Details and Statistics Feed Statistics
Preview: All Things Distributed

All Things Distributed

Werner Vogels' weblog on building scalable and robust distributed systems.

Last Build Date: Mon, 04 Sep 2017 06:47:44 PDT

Copyright: Copyright 2011

AI for everyone - How companies can benefit from the advance of machine learning

Mon, 04 Sep 2017 10:00:00 PDT

This article titled "Wie Unternehmen vom Vormarsch des maschinellen Lernens profitieren können" appeared in German last week in the "Digitaliserung" column of Wirtschaftwoche. When a technology has its breakthrough, can often only be determined in hindsight. In the case of artificial intelligence (AI) and machine learning (ML), this is different. ML is that part of AI that describes rules and recognizes patterns from large amounts of data in order to predict future data. Both concepts are virtually omnipresent and at the top of most buzzword rankings. Personally, I think – and this is clearly linked to the rise of AI and ML – that there has never been a better time than today to develop smart applications and use them. Why? Because three things are coming together. First: Users across the globe are capturing data digitally, whether this is in the physical world through sensors or GPS, or online through click stream data. As a result, there is a critical mass of data available. Secondly, there is enough affordable computing capacity in the cloud for companies and organizations, no matter what their size, to use intelligent applications. And thirdly, an "algorithmic revolution" has taken place, meaning it is now possible to train trillions of algorithms simultaneously, making the whole machine learning process much faster. This has allowed for more research, which has resulted in reaching the "critical mass" in knowledge that is needed to kick off an exponential growth in the development of new algorithms and architectures. We may have come a relatively long way with AI, but the progress came quietly. After all, during the last 50 years, AI and ML were fields that had only been accessible to an exclusive circle of researchers and scientists. That is now changing, as packages of AI and ML services, frameworks and tools are today available to all sorts of companies and organizations, including those that don't have dedicated research groups in this field. The management consultants at McKinsey expect that the global market for AI-based services, software and hardware will grow annually by 15-25% and reach a volume of around USD 130 billion in 2025. A number of start-ups are using AI algorithms for all things imaginable – searching for tumors in medical images, helping people learn foreign languages, or automating claims handling at insurance companies. At the same time, entirely new categories of applications are being created whereby a natural conversation between man and machine is taking center-stage. Progress through machine learning Is the hype surrounding AI and ML even justified? Definitely, because they offer business and society fascinating possibilities. With the help of digitization and high-performance computers, we are able to replicate human intelligence in some areas, such as computer vision, and even surpass the intelligence of humans. We are creating very diverse algorithms for a wide range of application areas and turning these individual pieces into services so that ML is available for everyone. Packaged into applications and business models, ML can make our life more pleasant or safer. Take autonomous driving: 90% of car accidents in the US can be traced to "human failure". The assumption is that the number of accidents will decline over the long term if vehicles drive autonomously. In aviation, this has already been reality for a long time. MIT pioneers Erik Brynjolfsson and Andrew McAfee predict that the macroeconomic effect of the so-called "second machine age" will be comparable to what the steam engine once unleashed when it replaced humans' muscular strength ("the first machine age"). Many are uncomfortable with the idea that an artificial intelligence exists alongside human intelligence. That is understandable. We must therefore discuss – parallel to the technological developments – how humans and AI can co-exist in the future; the moral and ethical aspects that arise; how to ensure we have a good grip on AI; and which l[...]

Improving Customer Service with Amazon Connect and Amazon Lex

Fri, 30 Jun 2017 11:00:00 PDT

Customer service is central to the overall customer experience that all consumers are familiar with when communicating with companies. That experience is often tested when we need to ask for help or have a question to be answered. Unfortunately, we've become accustomed to providing the same information multiple times, waiting on hold, and generally spending a lot more time than we expected to resolve our issue when we call customer service. When you call for customer assistance, you often need to wait for an agent to become available after navigating a set of menus. This means that you're going to wait on hold regardless of whether your issue is simple or complex. Once connected, the systems that power call centers generally don't do a good job of using and sharing available information. Therefore, you often start out anonymous and can't be recognized until you've gone through a scripted set of questions. If your issue is complex, you may end up repeating the same information to each person you talk to, because context is not provided with the handoff. It's easy to end up frustrated by the experience, even if your issue is successfully resolved. At Amazon, customer obsession is a fundamental principle of how we operate, and it drives the investments we make. Making sure that customers have a great experience when they need to call us is something that we've invested a lot of time in. So much so, that in March 2017, we announced Amazon Connect, which is the result of nearly ten years of work to build cloud-based contact centers at scale to power customer service for more than 50 Amazon teams and subsidiaries, including, Zappos, and Audible. The service allows any business to deliver better over-the-phone customer service at lower cost. When we set out to build Amazon Connect, we thought deeply about how artificial intelligence could be applied to improve the customer experience. AI has incredible potential in this area. Today, AWS customers are using the cloud to better serve their customers in many different ways. For instance, Zillow trains and retrains 7.5 million models every day to provide highly specific home value estimates to better inform buyers and sellers. KRY is helping doctors virtually visit patients and accurately diagnose aliments by applying machine learning to symptoms. Netflix is using machine learning to provide highly personalized recommendations to over 100 million subscribers. There are really exciting projects everywhere you look, including call centers. When Amazon Connect launched, we spoke about the integration with Amazon Lex. One of the really interesting trends in machine learning lately has been the rise of chatbots, because they are well suited to fulfilling customer requests with natural language. Amazon Lex, which uses the same conversational technology as Amazon Alexa, is Amazon Web Services' deep-learning powered chatbot platform. By linking Amazon Lex chatbots into the Amazon Connect contact flow, customers are able to get help immediately without relying on menus or specific voice commands. For example, an Amazon Lex driven conversation with your dentist's office might look like this… Connect: "Hello, thanks for calling. Is this Jeff?" Jeff: "Yes" Connect: "I see you have a cleaning appointment this Friday. Are you calling to confirm?" Jeff: "No, actually." Connect: "Ok, what are you calling about?" Jeff: "I'd like to change my appointment to be next Monday." Connect: "No problem, I have availability on Monday July 3rd at 11:00 AM. Does that work? Jeff: "That's fine." Connect: "Great. I have booked an appointment for you on Monday, July 3rd at 11:00 AM. Is there anything else I can help you with? Jeff: "Can you send me a text confirmation?" Connect: "Sure. I have sent a text message confirmation of your appointment to your cell. Can I do anything more for you?" Jeff: "No, that'[...]

Stop waiting for perfection and learn from your mistakes

Wed, 28 Jun 2017 10:00:00 PDT

This article titled "Wartet nicht auf Perfektion – lernt aus euren Fehlern!" appeared in German last week in the "Digitaliserung" column of Wirtschaftwoche. "Man errs as long as he doth strive." Goethe, the German prince of poets, knew that already more than 200 years ago. His words still ring true today, but with a crucial difference: Striving alone is not enough. You have to strive faster than the rest. And while there's nothing wrong with striving for perfection, in today's digital world you can no longer wait until your products are near perfection before offering them to your customers. If so, you will fall behind in your market. So if you can't wait for perfection, what should you do instead? I believe the answer is to experiment aggressively with your product development, accepting the possibility that some of your experiments will fail. Anyone who has listened to, or worked with, management gurus know their mantra: Failure is a necessary part of progress. That's true, but there's often a big gap between the management theory and the reality on the ground. People want to experiment and learn from things that go wrong. But in the flurry of day-to-day business, they're not given enough time to really reflect on the cause of an error and what to do differently next time. The solution is to find a systematic approach that prevents errors from repeating themselves. From perfection to anti-fragility In finding such a systematic way, you first need to distinguish between two types of errors that can happen in your company: those of technology and those of human decision-making. The nice thing is: if you know how to deal effectively with the first, you might end up being better in the second, making better decisions. The financial mathematician and essayist Nassim Taleb offers an interesting take on this issue. He has argued that errors are incredibly valuable because they lead to innovation. He uses the term 'anti-fragility' to make his point. Today's digital business models require smaller, frequent releases to reduce risk. That means the technologies underpinning these new business models must be more than just robust. They must be 'anti-fragile'. The main feature of anti-fragile technology is that it can 'err' without falling apart. In fact, a crisis can make it even better. At Amazon, we also require our systems and customer solutions to be anti-fragile, and we do that by designing our systems to stand the test of time. Our systems must be able to evolve and become more resilient to failure. They must become more powerful and more feature-rich over time as a result of learning from customer feedback and any failure modes they may encounter while operating the systems. An example of a German company that has become 'anti-fragile' is HARTING, the world's leading provider of heavy pluggable connectors for machines and plants. HARTING shows how to think a step ahead about the meaning of quality standards in the digital world. Quality and trust are the most important values for this traditional company, and Industry 4.0 and the digital transformation have already been important focus areas for them since 2011. Even though it was hard to accept at first, HARTING has meanwhile realized that errors are inevitable. For that reason, its development switched to agile methods. It also uses the "minimum-viable-product" approach and relies on microservices for its software. Working this way, HARTING can discard things and create new things more easily. All in all, HARTING has become faster. That can be seen with HARTING MICA, an edge computing solution that enables older machines and plants to get a digital retrofit. The body and hardware still reflect HARTING's standard of perfection. But for the software, the goal is "good enough", because a microservice is neither ever finished nor perfect. As a result, wrong decisions and mistakes can be corrected very quickly and[...]

Amazon DynamoDB Accelerator (DAX): Speed Up DynamoDB Response Times from Milliseconds to Microseconds without Application Rewrite.

Wed, 21 Jun 2017 11:00:00 PDT

Today, I'm excited to announce the general availability of Amazon DynamoDB Accelerator (DAX), a fully managed, highly available, in-memory cache that can speed up DynamoDB response times from milliseconds to microseconds, even at millions of requests per second. You can add DAX to your existing DynamoDB applications with just a few clicks in the AWS Management Console – no application rewrites required. DynamoDB has come a long way in the 5 years since we announced its availability in January 2012. As we said at the time, DynamoDB was a result of 15 years of learning in the area of large scale non-relational databases and cloud services. Based on this experience and learning, we built DynamoDB to be a fast, highly scalable NoSQL database to meet the needs of Internet-scale applications. DynamoDB was the first service at AWS to use SSD storage. Development of DynamoDB was guided by the core set of distributed systems principles outlined in the Dynamo paper, resulting in an ultra-scalable and highly reliable database system. DynamoDB delivers predictable performance and single digit millisecond latencies for reads and writes to your application, whether you're just getting started and want to perform hundreds of reads or writes per second in dev and test, or you're operating at scale in production performing millions of reads and writes per second. Saving crucial milliseconds Having been closely involved in the design and development of DynamoDB over the years, I find it gratifying to see DynamoDB being used by more than 100,000 customers - including the likes of AirBnB, Amazon, Expedia, Lyft, Redfin, and Supercell. It delivers predictable performance, consistently in the single-digit milliseconds, to users of some of the largest, most popular, iconic applications in use today. I've had a chance to interact with many of these customers on the design of their apps. These interactions allow me to understand their emerging needs, which I take back to our development teams to further iterate on our services. Many of these customers have apps with near real-time requirements for accessing data that need even faster performance than single-digit milliseconds. These are the apps that have motivated us to develop DAX. To give you some examples of my interactions, I've been talking to a few ad-tech companies lately, and their conversations are about how they can save milliseconds of performance. For their applications, they have 20-50 ms to decide whether or not to place a bid for an ad. Every millisecond that is spent querying a database and waiting for a key piece of data is time that they could otherwise use to make better decisions, process more data, or improve calculations to place a more accurate bid. These high-throughput, low-latency requirements need caching, not as a consideration, but as a best practice. Caches reduce latencies to microseconds, increases throughput, and in many cases, help customers save money by reducing the amount of resources they have to overprovision for their databases. Caching is not a new concept, and I have always wondered, why doesn't everyone cache? I think the reasons are many, but most follow a similar trend. Although many developers are aware of the patterns and benefits of adding a cache to an application, it's not easy to implement such functionality correctly. It's also time consuming and costly. When you write an application, you might not need or design for caching on day one. Thus, caching has to be shoehorned into an application that is already operational and experiencing load that would necessitate the added benefits. Adding caching when your app is already experiencing load is not easy. As a result, we see many folks trying to squeeze out every last drop of performance, or significantly overprovision their database resources to avoid adding a cache. Fully managed cache for DynamoDB What if you could seamlessly add caching to your application without requir[...]

Expanding the Cloud – An AWS Region is coming to Hong Kong

Wed, 21 Jun 2017 00:00:00 PDT

Today, I am very excited to announce our plans to open a new AWS Region in Hong Kong! The new region will give Hong Kong-based businesses, government organizations, non-profits, and global companies with customers in Hong Kong, the ability to leverage AWS technologies from data centers in Hong Kong. The new AWS Asia Pacific (Hong Kong) Region will have three Availability Zones and be ready for customers for use in 2018. Over the past decade, we have seen tremendous growth at AWS. As a result, we have opened 43 Availability Zones across 16 AWS Regions worldwide. Last year, we opened new regions in Korea, India, the US, Canada, and the UK. Throughout the next year, we will see another eight zones come online, across three AWS Regions (France, China, and Sweden). However, we do not plan to slow down and we are not stopping there. We are actively working to open new regions in the locations where our customers need them most. In Asia Pacific, we have been constantly expanding our footprint. In 2010, we opened our first AWS Region in Singapore and since then have opened additional regions: Japan, Australia, China, Korea, and India. After the launch of the AWS APAC (Hong Kong) Region, there will be 19 Availability Zones in Asia Pacific for customers to build flexible, scalable, secure, and highly available applications. As well as AWS Regions, we also have 21 AWS Edge Network Locations in Asia Pacific. This enables customers to serve content to their end users with low latency, giving them the best application experience. This continued investment in Asia Pacific has led to strong growth as many customers across the region move to AWS. Organizations in Hong Kong have been increasingly moving their mission-critical applications to AWS. This has led us to steadily increase our investment in Hong Kong to serve our growing base of enterprise, public sector, and startup customers. In 2008, AWS opened a point of presence (PoP) in Hong Kong to enable customers to serve content to their end users with low latency. Since then, AWS has added two more PoPs in Hong Kong, the latest in 2016. In 2013, AWS opened an office in Hong Kong. Today we have local teams in Hong Kong to help customers of all sizes as they move to AWS, including account managers, solutions architects, business developers, partner managers, professional services consultants, technology evangelists, start-up community developers, and more. Some of the most successful startups in the world—including 8 Securities, 9GAG, and GoAnimate—are already using AWS to deliver highly reliable, scalable, and secure applications to customers. 9GAG is a Hong Kong-based company responsible for, one of the top traffic websites in the world. It's an entertainment website where users can post content or "memes" that they find amusing and share them across social media networks. 9GAG generates millions of Facebook shares and likes per month, attracts over 78 million global unique visitors, and receives more than 1 billion page views per month. 9GAG has a small team of nine people, including three engineers to support the business, and uses AWS to service their global visitors. GoAnimate is a Hong Kong-based company that allows companies and individuals to tell great visual stories via its online animation platform. GoAnimate uses many AWS services, including Amazon Polly, to allow users to make their visual animations speak. They chose to use AWS in order to focus on developing their platform, instead of managing infrastructure. They believe that they have reduced development time from 20 to 30 percent by having done so. Some of the largest, and most well respected, enterprises in Hong Kong are also using AWS to power their businesses, enabling them to be more agile and responsive to their customers. These companies include Cathay Pacific, CLSA, HSBC, Gibson Innovations, Kerry Logistics, Ocean Park, Next Digital, and TownGas. Hong Kong[...]

Unlocking the Value of Device Data with AWS Greengrass.

Wed, 07 Jun 2017 09:00:00 PDT

Unlocking the value of data is a primary goal that AWS helps our customers to pursue. In recent years, an explosion of intelligent devices have created oceans of new data across many industries. We have seen that such devices can benefit greatly from the elastic resources of the cloud. This is because data gets more valuable when it can be processed together with other data. At the same time, it can be valuable to process some data right at the source where it is generated. Some applications – medical equipment, industrial machinery, and building automation are just a few – can't rely exclusively on the cloud for control, and require some form of local storage and execution. Such applications are often mission-critical: safeties must operate reliably, even if connectivity drops. Some applications may also rely on timely decisions: when maneuvering heavy machinery, an absolute minimum of latency is critical. Some use cases have privacy or regulatory constraints: medical data might need to be stored on site at a hospital for years even if also stored in the cloud. When you can't address scenarios such as these, the value of data you don't process is lost. As it turns out, there are three broad reasons that local data processing is important, in addition to cloud-based processing. At AWS we refer to these broad reasons as "laws" because we expect them to hold even as technology improves: Law of Physics. Customers want to build applications that make the most interactive and critical decisions locally, such as safety-critical control. This is determined by basic laws of physics: it takes time to send data to the cloud, and networks don't have 100% availability. Customers in physically remote environments, such as mining and agriculture, are more affected by these issues. Law of Economics. In many industries, data production has grown more quickly than bandwidth, and much of this data is low value. Local aggregation and filtering of data allows customers to send only high-value data to the cloud for storage and analysis. Law of the Land. In some industries, customers have regulatory or compliance requirements to isolate or duplicate data in particular locations. Some governments impose data sovereignty restrictions on where data may be stored and processed. Today, we are announcing the general availability of AWS Greengrass, a new service that helps unlock the value of data from devices that are subject to the three laws described above. AWS Greengrass extends AWS onto your devices, so they can act locally on the data they generate while still taking advantage of the cloud. AWS Greengrass takes advantage of your devices' onboard capabilities, and extends them to the cloud for management, updates, and elastic compute and storage. AWS Greengrass provides the following features: Local execution of AWS Lambda functions written in Python 2.7 and deployed down from the cloud. Local device shadows to maintain state for the stateless functions, including sync and conflict resolution. Local messaging between functions and peripherals on the device that hosts AWS Greengrass core, and also between the core and other local devices that use the AWS IoT Device SDK. Security of communication between the AWS Greengrass group and the cloud. AWS Greengrass uses the same certificate-based mutual authentication that AWS IoT uses. Local communication within an AWS Greengrass group is also secured by using a unique private CA for every group. Before AWS Greengrass, device builders often had to choose between the low latency of local execution, and the flexibility, scale, and ease of the cloud. AWS Greengrass removes that trade-off—manufacturers and OEMs can now build solutions that use the cloud for management, analytics, and durable storage, while keeping critical functionality on-device or nearby. AWS Greengrass makes it[...]

Weekend Reading: Amazon Aurora: Design Considerations for High Throughput Cloud-Native Relational Databases.

Fri, 19 May 2017 15:00:00 PDT


In many high-throughput OLTP style applications, the database plays a crucial role in achieving scale, reliability, high-performance, and cost efficiency. For a long time, these requirements were almost exclusively served by commercial, proprietary databases. Soon after the launch of the AWS Relational Database Service (RDS) customers gave us feedback that they would love to migrate to RDS. Yet, what they desired more, was a reality that unshackled them from the high-cost, punitive licensing schemes, which came with proprietary databases.

They would love to migrate to an open-source style database like MySQL or PostgreSQL, if such a database could meet the enterprise-grade reliability and performance these high-scale applications required.

We decided to use our inventive powers to design and build a new database engine that would give database systems such as MySQL and PostgreSQL reliability and performance at scale. Meaning, at a level that could serve even the most demanding OLTP applications. It gave us the opportunity to invent a new database architecture that would address to needs of modern cloud-scale applications, departing from the traditional approaches that had their roots in databases of the nineties. That database engine is now known as "Amazon Aurora" and launched in 2014 for MySQL, and in 2016 for PostgreSQL.

Amazon Aurora has become the fastest-growing service in the history of AWS and frequently is the target of migration from on-premise proprietary databases.

In a paper published this week at SIGMOD'17, the Amazon Aurora team presents the design considerations for the new database engine and how they addressed them. From the abstract:

Amazon Aurora is a relational database service for OLTP workloads offered as part of Amazon Web Services (AWS). In this paper, we describe the architecture of Aurora and the design considerations leading to that architecture. We believe the central constraint in high throughput data processing has moved from compute and storage to the network. Aurora brings a novel architecture to the relational database to address this constraint, most notably by pushing redo processing to a multi-tenant scaleout storage service, purpose-built for Aurora. We describe how doing so not only reduces network traffic, but also allows for fast crash recovery, failovers to replicas without loss of data, and fault-tolerant, self-healing storage. We then describe how Aurora achieves consensus on durable state across numerous storage nodes using an efficient asynchronous scheme, avoiding expensive and chatty recovery protocols. Finally, having operated Aurora as a production service for over 18 months, we share lessons we have learned from our customers on what modern cloud applications expect from their database tier.

I hope you will enjoy this weekend's reading, as it contain many gems about modern database design.

"Amazon Aurora: Design Considerations for HighThroughput Cloud-Native Relational Databases", Alexandre Verbitski, Anurag Gupta, Debanjan Saha, Murali Brahmadesam, Kamal Gupta, Raman Mittal, Sailesh Krishnamurthy, Sandor Maurice, Tengiz Kharatishvili, Xiaofeng Bao, in SIGMOD '17 Proceedings of the 2017 ACM International Conference on Management of Data, Pages 1041-1052 May 14 – 19, 2017, Chicago, IL, USA.

Faster, higher, stronger: How the digitalization of industry is redefining value creation

Wed, 03 May 2017 10:00:00 PDT

This article titled "Wie die Digitalisierung Wertschöpfung neu definiert" appeared in German last week in the "Größer, höher, weiter (bigger, higher, further)" column of Wirtschaftwoche. Germany's "hidden champions" – family-owned companies, engineering companies, specialists – are unique in the world. They stand for quality, reliability and a high degree of know-how in manufacturing. Hidden champions play a significant role in the German economy; as a result, Germany has become one of the few countries in Western Europe where manufacturing accounts for more than 20% of GDP. By contrast, neighboring countries have seen a continuous decline in their manufacturing base. What's more, digital technologies and business models that are focused on Industry 4.0 (i.e., the term invented in Germany to refer to the digitalization of production) have the potential to reinforce Germany's lead even more. According to estimates by Bitkom, the German IT industry association, and the Fraunhofer Institute of Industrial Engineering IAO, Germany's hidden champions will contribute a substantial portion to the country's economic growth by 2025 and create new jobs. At the same time, many experts believe the fundamental potential of Industry 4.0 has not even been fully leveraged yet. The power of persistence versus the speed of adjustment Most of Germany's hidden champions have earned their reputation through hard work: they have been optimizing their processes over decades. They have invested the time to perfect their processes and develop high-quality products for their customers. This has paid off – and continues to do so. However, digital technologies are now ushering in a paradigm change in value creation. Manufacturing can be fully digitalized to become part of a connected "Internet of Things" (IoT), controlled via the cloud. And control is not the only change: IoT creates many new data streams that, through cloud analytics, provide companies with much deeper insight into their operations and customer engagement. This is forcing Mittelstand companies to break down silos between departments, think beyond their traditional activities, and develop new business models. In fact, almost every industrial company in Germany already has a digitalization project in place. Most of these companies are extracting additional efficiency gains in their production by using digital technologies. Other companies have established start-ups for certain activities, or pilot projects aimed at creating showcases. But many of these initiatives never get beyond that point. The core business, which is doing well, remains untouched by all this. And one of the main reasons why is because the people with the necessary IT expertise in Mittelstand companies sometimes are not sitting at the strategy table as often as they should. Will these initiatives be enough to secure the pole position for Germany's Mittelstand? Probably not. Companies in growth markets are catching up. China's industry, for example, is making huge progress – something that took years to achieve in other places. The role of Chinese manufacturers in the worldwide market is changing: from low-cost workbench to global provider of advanced technology. Market leaders from Germany therefore realize they cannot afford to rest on their laurels. Competitors from the software side are also reshuffling the balance of power, because their offerings will create a completely new market alongside the traditional business of Mittelstand toolmakers and mechanical and systems engineering companies. If new and innovative companies, such as providers of data analytics, specialized software providers or companies that can bundle complementary offerings, appear on the scene, traditional manufacturing would suddenly become just one module among many – namely ma[...]

Coming to STATION F: The first Mentor's Office powered by AWS!

Wed, 12 Apr 2017 01:00:00 PDT

I am excited to announce that AWS is opening its first Mentor's Office at STATION F in Paris! The Mentor's Office is a workplace exclusively dedicated to meetings between AWS experts and the startups. STATION F is the world's biggest startup campus. With this special offer starting at the end of June, at the campus opening, AWS increases the support already available to startup customers in France. All year long, AWS experts will deliver technical and business assistance to startups based on campus. AWS Solutions Architects will meet startup members for face-to-face sessions, to share guidance on how cloud services can be used for their specific use cases, workloads, or applications. Startup members will also have the possibility to meet with AWS business experts such as account managers, business developers, and consultants. They can explore the possibilities of the AWS Cloud and take advantage of our IT experience and business expertise. With these 1:1 meetings, AWS delivers mentoring to startups to help them bring their ideas to life and accelerate their business using the cloud. AWS will also provide startups with all of the benefits of the AWS Activate program, including AWS credits, training, technical support, and a special startup community forum to help them successfully build their business. For more details about the Mentor's Office at STATION F, feel free to contact the AWS STATION F team. With this opening, Amazon continues to build out global programs to support startup growth and to speed up innovation. Startups can also apply to other Amazon programs to boost their businesses, such as: Amazon Launchpad, which makes it easy for startups to launch, market, and distribute their products to hundreds of millions of Amazon customers across the globe. Alexa Fund, which provides up to $100 million in venture capital funding to fuel voice technology innovation. After the launch of AWS in 2006, we saw an acceleration of French startups adopting the cloud. Successful French startups already using AWS to grow their businesses, across Europe and around the world, include Captain Dash, Dashlane, Botify, Sketchfab,Predicsis, Yomoni, BidMotion, Teads, FrontApp, Iconosquare, and many others. They all get benefits from AWS's highly flexible, scalable, and secure global platform. AWS eliminates the undifferentiated heavy lifting of managing underlying infrastructure and provides elastic, pay-as-you-go IT resources. We have also seen start-ups in France using AWS to grow and become household names in their market segment, such as Aldebaran Robotics (SoftBank Robotics Europe). This startup uses AWS to develop new technologies. They are able to concentrate their engineering resources on innovation, rather than maintaining technology infrastructure, which is leading to the development of autonomous and programmable humanoid robots. Cloud is also an opportunity for startups to reach security standards that were not accessible before. For example, PayPlug is an online payment by credit card solution enabling e-merchants to enrich the customer experience by reinventing the payment experience. Such a service requires suppliers to get PCI DSS certification for the "Service Provider" level, a very demanding certification level. Using AWS's PCI DSS Level 1 compliant infrastructure, Payplug has been certified by L'ACPR (L'Autorité de contrôle prudentiel et de resolution, the French supervisory for prudential and resolution authority) as a financial institution, a major step in their development. I look forward to meeting the builders of tomorrow at STATION F in the near future. Go French Startups! [...]

Back-to-Basics Weekend Reading: Twenty years of functional MRI: The science and the stories.

Fri, 07 Apr 2017 09:00:00 PDT


I will be returning this weekend to the US from a very successful AWS Summit in Sydney, so I have ample time to read during travels. This weekend however I would like to take a break from reading historical computer science material, to catch up on another technology I find fascinating, that of functional Magnetic Resonace Imaging, aka fMRI.

fMRI is a functional imagine technology, meaning that it just records the state of the brain at one particular point in time, but the changing state over a period of time. The basic technology records brain activity by measuring changes in blood flow through the brain. The technology relies on the fact that cerebral blood flow and neuronal activation are coupled. When an area of the brain is in use, blood flow to that region also increases.

There have been significant advances in the use of fMRI technology, but mostly in research. It also comes with significant ethical questions: if you can "read" someone's brain, what are you allowed to do what that knowledge?

For my flight back to the US this weekend I will read two papers: one by Peter Bandettini pubslished in NeruImagine about the history of fMRI and one from Poldrack and Farah on the state of the art in fMRI and its applications, published in Nature.

"Twenty years of functional MRI: The science and the stories, Peter A. Bandettini, Neuroimage 62, 575–588 (2012)

"Progress and challenges in probing the human brain", Russell A. Poldrack and Martha J. Farah, Nature 526, 371–379 (15 October 2015)

Välkommen till Stockholm – An AWS Region is coming to the Nordics

Tue, 04 Apr 2017 00:00:00 PDT

Today, I am very excited to announce our plans to open a new AWS Region in the Nordics! The new region will give Nordic-based businesses, government organisations, non-profits, and global companies with customers in the Nordics, the ability to leverage the AWS technology infrastructure from data centers in Sweden. The new AWS EU (Stockholm) Region will have three Availability Zones and will be ready for customers to use in 2018. Over the past decade, we have seen tremendous growth at AWS. As a result, we have opened 42 Availability Zones across 16 AWS Regions worldwide. Last year, we opened new regions in Canada, India, Korea, the UK, and the US. Throughout the next year we will see another five zones, across two AWS Regions, come online in France and China. However, we do not plan to slow down and we are not stopping there. We are actively working to open new regions in the locations our customers need them most. In Europe, we have been constantly expanding our footprint. In 2007, we opened our first AWS Region in Ireland and since then have opened additional regions, in Germany and the UK, with France still to come. After the launch of the AWS EU (Stockholm) Region, there will be 13 Availability Zones in Europe for customers to build flexible, scalable, secure, and highly available applications. It will also give customers another region where they can store their data with the knowledge that it will not leave the EU unless they move it. As well as AWS Regions, we also have 24 AWS Edge Network Locations in Europe. This enables customers to serve content to their end users with low latency, giving them the best application experience. This continued investment in Europe has led to strong growth as many customers across the region move to AWS. Organizations across the Nordics—Denmark, Finland, Iceland, Norway, and Sweden—have been increasingly moving their mission-critical applications to AWS. This has led us to steadily increase our investment in the Nordics to serve our growing base of enterprise, public sector, and startup customers. In 2011, AWS opened a Point of Presence (PoP) in Stockholm to enable customers to serve content to their end users with low latency. In 2014 and 2015 respectively, AWS opened offices in Stockholm and Espoo, Finland. We have also added teams in the Nordics to help customers of all sizes as they move to AWS, including account managers, solutions architects, business developers, partner managers, professional services consultants, technology evangelists, start-up community developers, and more. Some of the most successful startups in the world, including Bambora, iZettle, King, Mojang, and Supercell are already using AWS to deliver highly reliable, scalable, and secure applications to customers. Supercell is responsible for several of the highest grossing mobile games in history, and they rely on AWS for their entire infrastructure. With titles like Boom Beach, Clash of Clans, Clash Royale, and Hay Day, Supercell has 100 million people playing their games every single day. iZettle, a mobile payments startup, is also ‘all-in’ on AWS. After finding it cost prohibitive to use colocation centers in local markets where their users are based, iZettle decided to give up hardware. They migrated their IT infrastructure, including mission-critical payments platforms, to AWS in just six weeks. After migrating, database queries that took six seconds now take three seconds in their AWS infrastructure. That’s 100% faster. Some of the largest, and most well respected, enterprises in the Nordics also depend on AWS to power their businesses, enabling them to be more agile and responsive to their customers. These companies include ASSA ABLOY, Finnair, Husqvarna Group, IKEA, Kauppalehti, Kesko, Sanoma, Scania, Sc[...]

Back-to-Basics Weekend Reading: An Implementation of a Log-Structured File System

Thu, 30 Mar 2017 09:00:00 PDT


This weekend I am travelling to Australia for the first AWS Summit of 2017. I find on such a long trip, to keep me from getting distracted, I need an exciting paper that is easy to read. Last week's 'Deep Learning' overview would have not met those requirements.

One topic that always gets me excited is how to take computer science research and implement it in production systems. There are often so many obstacles that we do not see much of this work happening. For example when building Dynamo, where we put a collection of different research technologies together in production, we struggled with all the assumptions the researchers had made. At times, it makes research unsuitable for production (e.g. real systems do not fail by stopping in a nice and clean way).

In the early nineties, Mendel Rosenblum and John Ousterhout had made a major breakthrough in the design of file systems with "The Design and Implementation of a Log-Structured File System." That alone is an interesting paper to read, but this weekend we will be looking at the actual implementation of an LFS by Margo Seltzer and other members of the BSD team.

It is one of the first papers to describe the implementation of a research system, and measure the result within a production system. I hope you will also enjoy it!

"An implementation of a log-structured file system for UNIX.", Margo Seltzer, Keith Bostic, Marshall Kirk Mckusick, and Carl Staelin. 1993, In Proceedings of the USENIX Winter 1993 Conference Proceedings on USENIX Winter 1993 Conference Proceedings (USENIX'93). USENIX Association, Berkeley, CA, USA, 3-3.

Back-to-Basics Weekend Reading: Deep learning in neural networks

Fri, 24 Mar 2017 21:00:00 PDT


In the past few years, we have seen an explosion in the use of 'Deep Learning' as its software platforms and the supporting hardware mature, especially as GPUs with larger memories become widely available. Even though this is a recent development, 'Deep Learning' has entrenched historical roots, tracing back all the way to the sixties or possibly earlier.

By reading-up on its history, we get a better understanding of the current state of the art of 'Deep Learning algorithms' and the 'Neural Networks' that you build with them.

There is a broad set of papers to read if we want to dive deep into the history. It would take us multiple weekends. Instead, we will be reading an excellent overview paper from 2014 by Jürgen Schmidhuber. Jürgen evaluates the current state of the art in 'Deep Learning' by tracing it back to its roots. Ergo, we get excellent historical context.


"Deep Learning in Neural Networks: An Overview." Jürgen Schmidhuber, in Neural Networks, Volume 61, January 2015, Pages 85-117 (DOI: 10.1016/j.neunet.2014.09.003)

Amazon Makes it Free for Developers to Build and Host Most Alexa Skills Using AWS

Wed, 15 Mar 2017 10:00:00 PDT

Amazon today announced a new program that will make it free for tens of thousands of Alexa developers to build and host most Alexa skills using Amazon Web Services (AWS). Many Alexa skill developers currently take advantage of the AWS Free Tier, which offers one million AWS Lambda requests and up to 750 hours of Amazon Elastic Compute Cloud (Amazon EC2) compute time per month at no charge. However, if developers exceed the AWS Free Tier limits, they may incur AWS usage fees each month. Now, developers with a live Alexa skill can apply to receive a $100 AWS promotional credit and can earn an additional $100 per month in AWS promotional credits if they incur AWS usage charges for their skill – making it free for developers to build and host most Alexa skills. Our goal is to free up developers to create more robust and unique skills that can take advantage of AWS services. We can't wait to see what you create. How It Works If you have one or more live Alexa skills, you are eligible to receive a $100 AWS promotional credit to be used toward AWS fees incurred in connection with your skills. Additionally, if you continue to incur skill-related AWS charges that exceed the initial $100 promotional credit, you will also be eligible to receive monthly AWS promotional credits of $100. All you need to do is apply once. Apply Now > Build and Host Alexa Skills with AWS With the new program, if you exceed the AWS Free Tier due to growth of your skill, or are looking to scale your skill using AWS services, you will be eligible to receive AWS promotional credits to be applied to AWS services such as Amazon EC2, Amazon Simple Storage Service (Amazon S3), Amazon DynamoDB, and Amazon CloudFront. For example, you can use DynamoDB to create more engaging skills that have context and memory. In a game with memory, you could pause for a few hours and then keep going (like the Wayne Investigation, or Sub War). Or, to give your customers a more immersive experience, consider incorporating audio files via Amazon S3 to stream short audio bursts, games, podcasts, or news stories in your skill. Many of our most engaging skills, like Ambient Noise and RuneScape Quests – One Piercing Note, add audio sounds to soothe and voiceovers and sound effects to make the in-game experience more immersive. Build a Skill Today - Special Offers Our skill templates and step-by-step guides are a valuable way to quickly learn the end-to-end process for building and publishing an Alexa skill. You can get started quickly with the city guide template or fact skill template, or use the Alexa SDK for Node on GitHub to create a custom skill. Plus, if you publish a skill, you'll receive an Alexa dev t-shirt. Quantities are limited. See Terms and Conditions. Additional Resources For more information on getting started with developing for Alexa, check out the following resources: Voice Design Best Practices Alexa Skills Kit (ASK) Alexa Voice Service (AVS) The Alexa Fund ASK Developer Forums Weekly Developer Office Hours [...]

How companies can become magnets for digital talent

Mon, 13 Mar 2017 10:00:00 PDT

This article titled "Wie Unternehmen digitale Talente anziehen" appeared in German last week in the "Tipps für Arbeitgeber" section of Wirtschaftwoche. The rise in digital business models is a huge challenge for recruiting and talent selection. The sort of skills businesses need today are in short supply. How companies can prepare themselves to attract the best talents for shaping their digital business. Digitalization offers almost endless possibilities to communicate faster, work more efficiently, and be more creative – in real-time. But groundbreaking digital business models need pioneers: creators, forward-looking thinkers and inventors who don't hesitate to leave the beaten path, embody ownership, and who understand how to translate customers' wishes into superb new products, services and solutions that evolve with speed. It is a no-brainer, that getting the right talent on board can decisively accelerate a company's digital transformation. At the same time, if your daily corporate practice doesn't fulfill their expectations regarding a vibrant and flexible working culture and a social media-minded environment, digital natives will simply turn their back on you and go elsewhere. Finding those kind of people is not easy. There are probably only a few companies that can say, they already have a sufficient number of such employees among their staff. Job openings for machine learning scientists, data analytics experts, IT security experts or developers are already difficult to fill, and the demand for this knowledge will increase significantly in the next few years as customers show their demand for digital engagements. The market for digital skills is "hot", in the U.S. as well as in Germany. And these talents are by no means coveted only by companies that always had a digital business model to begin with; suppliers to the automotive industry, financial services companies, and retailers also, urgently need product managers, and technical staff who can quickly make their organizations digitally attractive to their customers. Recruiting and selection in the digital age therefore needs to be tackled in a more strategic way than in the past. So how do you position your company as an attractive employer for digital talent? Preparing the organization for a new beginning One way is to eliminate rigid structures, previously the enemy to digital thinking. Digitalization involves, among others, suddenly converging areas that used to be siloed. Take industrial companies. In the past, their sales departments defined specifications according to the customer's wishes, which were then transferred step by step into the manufacturing process. These days, it's expected that everything should happen almost simultaneously. Previously, the top priorities for IT departments were equipping data centers with hardware, purchasing software, and further developing proprietary software. Today, companies take their server capacity and software from the cloud. These changes have to be taken into account when scanning the market for talent. At Düsseldorf-based fashion retailer Peek&Cloppenburg, for example, the business, development and IT functions are increasingly cooperating with each other because they realize that isolated departments and rigid hierarchies can slow down the organization's innovative strength and speed. That is also why employees have more and more room to make decisions themselves. P&C's digital transformation is supported by an in-house consulting team that helps the specialized departments analyze and digitize those processes that strengthen the customer touchpoints. The freedom to create A[...]

Back-to-Basics Weekend Reading: The Foundations of Blockchain

Fri, 10 Mar 2017 08:00:00 PST


More and more we see stories appearing, like this one in HBR by MIT Media Lab's Joi Ito and crew. It praises the power of blockchain as a disruptive technology, on par with how "the internet" changed everything.

I am always surprised to see that these far-reaching predictions are made, without diving into the technology itself. This weekend I would like to read about some of the technologies that predate blockchain, as they are its fundamental building blocks.

Blockchain technology first came on the scene in 2008, as a core component of the bitcoin cryptocurrency. Blockchain provides transactional, distributed ledger functionality that can operate without a centralized, trusted authority. Updates recorded in the ledger are immutable, with cryptographic time-stamping to achieve serializability. Blockchain's robust, decentralized functionality is very attractive for global financial systems, but can easily be applied to contracts, or operations such as global supply chain tracking.

When we look at the foundation of blockchain, there are three papers from the nineties that describe different components whose principles found its way into blockchain. The 91 paper by Haber and Stornetta describes how to use crypto signatures to time-stamp documents. The 98 paper by Schneier and Kelsey describes how to use crypto to protect sensitive information in log files on untrusted machines. Finally, the 96 paper by Ross Anderson describes a decentralized storage system, from which recorded updates cannot be deleted.

I hope these will enlighten your fundamental understanding of blockchain technology.

"How to Time-Stamp a Digital Document", Stuart Haber, and W. Scott Stornetta, In Advances in Cryptology – Crypto ’90, pp. 437–455. Lecture Notes in Computer Science v. 537, Springer-Verlag, Berlin 1991.

"Cryptographic Support for Secure Logs on Untrusted Machines", Bruce Schneier, and John Kelsey, in The Seventh USENIX Security Symposium Proceedings, pp. 53–62. USENIX Press, Januar 1998.

"The Eternity Service", Ross J. Anderson. Pragocrypt 1996.

Back-to-Basics Weekend Reading: Why Do Computers Stop and What Can Be Done About It?

Sat, 04 Mar 2017 09:00:00 PST


"Everything fails, all the time." A humble computer scientist once said. With all the resources we have today, it is easier for us to achieve fault-tolerance than it was many decades ago when computers began playing a role in critical systems such as health care, air traffic control and financial market systems. In the early days, the thinking was to use a hardware approach to achieve fault-tolerance. It was not until the mid-nineties that software fault-tolerance became more acceptable.

Tandem Computer was one of the pioneers in building these fault-tolerant, mission-critical systems. They used a shared-nothing multi-cpu approach. This is where each CPU had its own memory- and io-bus, and all were connected through a replicated shared bus, over which the independent OS instances could communication and run in lock step. In the late seventies and early eighties, this was considered state of the art in fault-tolerance.

Jim Gray, the father of concepts like transactions, worked for Tandem on software fault-tolerance. To be able to build better systems, he went deep in deconstructing the kind of failures Tandem customers were experiencing. He wrote up his findings in his "Why do Computers Stop" report. For a very longtime, this would be the only study available on reliability in production computer systems.

As important as the study is, the paper additionally covers "What can be done about it." Jim, for the first time, introduces concepts like process-pairs and transactions as the basis for software fault-tolerance. This is one of the fundamental papers of fault-tolerance in distributed systems, and I am going to enjoy reading it this weekend. I hope you will also.

"Why Do Computers Stop and What Can Be Done About It?", Jim Gray, June 1985, Tandem Technical report 85.7

Back-to-Basics Weekend Reading: Byzantine Generals

Fri, 24 Feb 2017 20:00:00 PST


In Reliable Distribution Systems, we need to handle different failure scenarios. Many of those deal with message loss and process failure. However, there is a class of scenarios that deal with malfunctioning processes, which send out conflicting information. The challenge is developing algorithms that can reach an agreement in the presence of these failures.

Lamport described that he was frustrated with the attention that Dijkstra had gotten for describing a computer science problem as the story of dining philosophers. He decided the best way to attract attention to a particular distributed systems problem was to present it in terms of a story; hence, the Byzantine Generals.

Abstractly, the problem can be described in terms of a group of generals of the Byzantine army, who camped with their troops around an enemy city. Communicating only by messenger, the generals were required to agree upon a common battle plan. However, one or more of them may be traitors who would try to confuse the others. The problem is: to find an algorithm that ensures the loyal generals will reach an agreement.

It is shown, using only oral messages, this problem is solvable if and only if more than two-thirds of the generals are loyal. So, a single traitor can confound two loyal generals. With unforgeable written messages, the problem is solvable for any number of generals and possible traitors. This weekend, I will be going back in time and reading three fundamental papers that laid-out the problems, and its first solutions. In the SIFT paper, the problem is first described, the "reaching agreement" paper describes the fundamental 3n+1 processor solution, and the last paper reviews and generalizes the previous results.

Maybe you will enjoy them as well. "SIFT: Design and Analysis of a Fault-Tolerant Computer for Aircraft Control" John H. Wensley, Leslie Lamport, Jack Goldberg, Milton W. Green, Karl N. Levitt, P. M. Melliar-Smith, Robert E. Shostak, Charles B. Weinstock, in Proceedings of the IEEE 66, October 5, 1978

"Reaching Agreement in the Presence of Faults" M. Pease, R. Shostak, and L. Lamport, 1980, J. ACM 27, 2 (April 1980), 228-234.

"The Byzantine Generals Problem", Lamport, L.; Shostak, R.; Pease, M. (1982), ACM Transactions on Programming Languages and Systems. 4 (3): 382–401. doi:10.1145/357172.357176.

Back-to-Basic Weekend Reading: Monte-Carlo Methods

Fri, 10 Feb 2017 11:00:00 PST


I always enjoy looking for solutions to difficult challenges in non-obvious places. That is probably why I like using probabilistic techniques for problems that appear to be hard, or impossible to solve deterministically. The probabilistic approach may not result in the perfect result, but it may get you very close, and much faster than deterministic techniques (which may even be computationally impossible).

Some of the earliest approaches using probabilities in physics experiments resulted in the Monte Carlo methods. Their essential idea is using randomness to solve problems that might be deterministic in principle. These are a broad class of computational algorithms that rely on repeated random sampling to obtain numerical results.

The Monte Carlo methods can be traced back to Stanislaw Ulam, John van Neuman, and Nick Metropolis at the Los Alamos Scientific Laboratory in the late 40s. The Monte Carlo methods were crucial in the simulations of the Manhattan Project, given the limited computational power available in those days

The paper I will be reading this weekend is the original paper from 1949, by Metropolis and Ulam. For fun, I’ve also decided to add a second paper by Herbert Anderson, who was a member of the Manhattan project. Anderson’s paper describes the use of Monte Carlo methods, and the computers in the Manhattan project.

The Monte Carlo Method”, Nicholas Metropolis, S. Ulam, Journal of the American Statistical Association, Vol. 44, No. 247. (Sep., 1949), pp. 335-341.

"Metropolis, Monte Carlo and the MANIAC", Anderson, Herbert L., Los Alamos Science, (1986) 14: 96–108.

Back-to-Basics Weekend Reading - Bloom Filters

Fri, 03 Feb 2017 11:00:00 PST


Listening to the "Algorithms to Live By" audio on my commute this morning, once again I was struck by the beauty of Bloom Filters. So, I decided it is time to resurrect the 'Back-to-Basics Weekend Reading' series, as I will be re-reading some fundamental CS papers this weekend.

In the past, I have done some weekend reading about Counting Bloom Filters, but now I am going even more fundamental, and I invite you to join me.

Bloom Filters, conceived by Burton Bloom in 1970, are probabilistic data structures to test whether an item is in a set. False positives are possible, but false negatives are not. Meaning, if a bit in the filter is not set, you can be sure the item is not in the set. If it is in the set, the mapped item may be in the set.

This is a hugely important technique if you need to process and track massive amounts of unique data units, as it is very space-efficient. From Dynamo and Postgresql, to HBase and Bitcoin, Bloom Filters are used in almost all modern distributed systems. This weekend I will be reading the original paper by Bloom from 1970, and another more recent survey paper that describes several variants and applications that have been developed over the years.

"Space/Time Trade-offs in Hash Coding with Allowable Errors", Bloom, Burton H., in Communications of the ACM, 13 (7): 422–426

"Cache-, Hash- and Space-Efficient Bloom Filters", Putze, F.; Sanders, P.; Singler, J., in Demetrescu, Camil, Experimental Algorithms, 6th International Workshop, WEA 200