Subscribe: Bill St. Arnaud
Added By: Feedage Forager Feedage Grade B rated
Language: English
cloud  clouds  commercial clouds  commercial  content  data  infrastructure  internet  network  networks  research  services 
Rate this Feed
Rate this feedRate this feedRate this feedRate this feedRate this feed
Rate this feed 1 starRate this feed 2 starRate this feed 3 starRate this feed 4 starRate this feed 5 star

Comments (0)

Feed Details and Statistics Feed Statistics
Preview: Bill St. Arnaud

Bill St. Arnaud

Future Internet, R&E Networks, Green Internet, Green IT

Updated: 2017-06-01T19:23:23.861-07:00


Green Bond Funds and the role of R&E networks


Around the world universities, R&E networks and researchers in general are looking at an increasingly austere future of budget cutbacks and reduced funding for higher education.  This environment is unlikely to change, even if the current global economic situation improves.  Healthcare, long term debt and aging populations are continue to put enormous pressure on government budgets for the foreseeable future.  In times of budget constraints education and research usually always gets the short shrift, even though it is argued that it is an investment in our future economic growth.The only bright spot in terms of new funding are various public and private sector programs related to climate change.  Even “denialist” governments as in Ottawa, Washington and now Canberra have launched several programs to fund new initiatives in energy, GHG abatement,  green programs etc.  The private financial sector has also started to launch several new instruments in this area with the development of Green Bonds and revolving funds.  Many of these funds are also supported by generous tax breaks.Green Bond, revolving funds and related initiatives are usually used to underwrite the capital costs of large renewable energy or energy efficiency projects.  Examples include deploying large solar arrays, wind farms, etc.  The funds earn their return on investment through the payback on energy savings or from feed in tariffs to the electrical grid.Green bonds in the United States got a major boost from the America Jobs Creation Act of 2004. It was designed to provide funding - in the form of $2 billion worth of AAA-rated bonds issued by the United States Treasury - to finance environmentally friendly development. Many individual states are also issuing Green Bond funds.  Some countries like the Netherlands Green Fund programs also fund innovation through the use of these funds.There is a significant opportunity for universities and R&E networks to tap into this programs.  But the challenge for most universities is their relative small size and lack of experience or knowledge in negotiating with Green Bond brokers.  Most Green Bond and Revolving funds are several hundred million dollars in size.  It is very hard for even a large university to come up with green projects of sufficient size to attract the attention of such investment vehicles.  This is where R&E networks can play an important role, by partnering with Green Bond brokers to aggregate demand from many institutions to put together a sizable enough package to attract large institutional investors such as pension funds etc.  In essence it is very similar to offering a Net+ service to the institutions.  Many universities, particularly in the US, are also part of a $1 billion green revolving fund which can be leveraged in a similar manner.To this point there seems no obvious role for the R&E network, other than aggregating demand.  But this could be argued could be done just as easily by the university business offices or  associations of university financial officers and/or presidents.  The big advantage that R&E networks bring is significantly increasing the project’s return on investment of the green bond or revolving fund by leveraging Net+ services to reduce a university’s energy footprint.  Computing and networking represent anywhere from 25-40% of an institution’s electrical budget.    Every initiative that reduces or eliminates that electrical consumption footprint, can be applied directly to improving the ROI of the green bond investment, either to the benefit of the investors or better as an additional revenue stream to the institution and R&E network.  And where Green Bonds also fund innovation, the use of ICT to reduce an institution’s environmental footprint can be significant.I have blogged many times in the past about ICT technologies that can reduce electrical energy consumption on the campus such as the use of commercial clouds[...]

Critical Future Role for R&E networks is to fund university IT departments to help reduce costs


I have doing some consulting work for a number of R&E networks around the world and the most common issue facing all of them is the looming financial constraints facing their institutions.  As reported in the Chronicle  of Higher Education campus networks are being squeezed as demand rises and budgets shrink .  The explosion of wireless devices, cloud computing, research computing and services for dormitories are putting increased pressure on most campus IT departments.A number of R&E networks are starting to address these problems of their member institutions by offering hosting services, outsourcing network support functions etc.  But two important areas where R&E networks can play a critical role in helping their member institutions is by deploying in depth content delivery networks and integrated WiFi/4G solutions.  One of the biggest costs for many IT departments is transit or commodity Internet. Diverting this traffic to integrated content delivery networks can reduce transit traffic by as much as 90%.   In North America Netflix can represent 50% of the traffic on some campuses with large residence student population. The other big cost center is managing the hundreds, if not thousands of WiFi hotspots and integrating these with new 4G services from commercial carriers. Several R&E networks have expanded Eduroam to local restaurants and coffee shops, while others are offloading local Wifi/4G management and making it a national service across the country.But I think the most important area  where R&E networks can play a critical role is to fund IT cost saving measures at member institutions.  The local IT staff have probably the best knowledge where savings can be obtained if they had sufficient money to invest in new technologies and management systems.  Funding of IT departments at institutions by R&E networks does not require a massive infusion of money from government.  We already have such a capital/cost saving model in place called  the University President’s 1$ billion Challenge Green Fund.  The $1 Billion Green Challenge Fund provides funding to universities and other nonprofits that finances energy efficiency upgrades on campuses. Harvard, Stanford, and other leading universities in Canada and the US have  committed over $65 million to finance upgrades in energy efficiency at participating institutions. The Challenge is inspired by the exceptional performance of existing revolving funds, which have a median annual return on investment of 32%. Revolving funds are often part of a  university endowment program or publicly traded entities.Although many revolving funds are focused on energy efficiency such as building insulation, new boilers etc, there is no reason why this same model could not be used to fund investments in reducing IT costs on campus.  R&E networks could also establish revolving funds in partnership with institutions where cost savings enabled by the R&E network, described above, could be used as a source of revenue to be shared with the IT department and the R&E network, rather than the current model of further taxing membership budgets at the institutions. Another variant on this concept  I have proposed is a “refundable” membership model where both the institution and R&E network jointly explore cost saving scenarios and share in their benefit.  Refundable membership models acts as an incentive for the institution to explore and find these cost savings.In the coming years institutional IT departments and R&E networks are going to have to find new business models, as they will be under increasing pressure to reduce costs, despite the growth in demand for their services.  Charging institutions fee for service or flat membership fees is unlikely to be sustainable in the coming years.$1 Billion University Green program launched - CIO and NRENs could be big beneficiary[...]

How to meaure economic impact of R&E Networks and Cyber-infrastructure


[One of the major challenges for government is to measure the economic impact of academic research.
In terms of research impact there are the usual standard indicators of citations, papers published, patents filed and number of graduate students. But governments and funding agencies usually want to measure direct economic impact in terms of jobs, new businesses and commercialization of research. Although there are a number of programs to promote commercialization of research the impacts and outcomes, to date, are a best a mixed result. There is a lot of hand waving as to the actual number of jobs and business opportunities such programs have created.
An alternate approach to measure the impact of academic research was undertaken by MIT which measured the economic impact of companies founded by MIT alumni, based on one of the largest surveys of entrepreneur alumni ever conducted. It was estimated that at the end of 2006, there were 25,600 active companies founded by living MIT alumni, employing 3.3 million people and generating annual world revenues of nearly $2 trillion. This group of companies, if its own nation, would be the 11th-largest economy in the world.
See for more details.
This approach of measuring impact reflects the well known truism that the biggest transfer of knowledge from academia to industry occurs once a year at graduation. Another good example is a landmark study undertaken by University of Toronto researchers which showed that the adoption and growth of the commercial Internet was driven in early stages by recently graduated students who had been exposed to the benefits of the Internet at their respective universities and community colleges. “The (Teaching) Role of Universities in the Diffusion of the Internet”

Measuring the economic activities of students who have been exposed to the latest research developments is probably the most effective way of understanding the impact of academic research. But other than the MIT and UoT studies few other research organizations have undertaken such a methodology to measuring the impact of academic research. There is no question it would be a difficult and expensive undertaking to locate past alumni and determine their contributions to society not only in the creation of new businesses but also indirectly in the improvements they make to companies they may work for. However, with new social networking tools like LinkedIn, ResearchGate, etc combined with academic collaboration tools such COmanage or SURFconext it may be possible to develop apps that cross reference entrepreneurial outcomes with sources of inspiration of academic research. This is a classic “Big Data” challenge.

This approach would not only measure academic research impact but also the contributions made by research networks and cyber-infrastructure, which are often forgotten in the larger scheme of things. It must be noted that most of the major contributions to the Internet such as Google, Facebook, etc were by students at universities and in dormitories who had access to the early unfettered Internet. -- BSA]

R&E Network and Green Internet Consultant.
twitter: BillStArnaud
skype: Pocketpro

Up to 98% of Internet traffic now consists of content that can be cached locally on servers


[It is interesting to see this report from Analysys Mason that confirms my analysis several years ago that most Internet traffic is moving to the edge delivered by Content Distribution Networks (CDN) delivered at Internet Exchange Points (IXPs) around the world. “In a paper released yesterday, UK analyst firm Analysys Mason estimates that 98 percent of internet traffic now consists of content that can be stored on servers, such as streaming video or web pages. These servers can be located in multiple locations around the world, and then delivered to users faster and at lower cost. The result is a shift in usage patterns and global Internet traffic flows. This combined with deeper penetration of IXPs and caching means that the way traffic flows across networks is changing too. The paper was written to persuade governments that the proposed ITU regulatory changes would hinder the growth of the web, but the report is well worth reading as a way of understanding how the web has changed over time. For example, 70% of international Internet bandwidth originating in Africa went to the USA in 1999, but by 2011 this figure had plunged to less than 5% as bandwidth shifted to Europe. Now, content is increasingly being stored on servers in Africa, where it can be accessed domestically or regionally.”“Applying unwarranted static voice regulations to the dynamic Internet would negatively impact users across the globe and slow or reverse current growth trends. Furthermore, the rate regime system would be difficult to design and expensive to implement, and even then would increase the cost of content delivery and hinder network investment at the expense of end users.”It is not only CDNs but commercial clouds and social software services like Facebook, Twitter, etc are part of this evolution. This evolution in Internet traffic will have a major impact on Internet architectures, addressing and naming. Please see for more details. Research and Education networks can play a critical role in developing open standards for CDN networks to distribute research, education and public broadcast TV and radio. Please see my previous blog on this subject For example At the last NANOG (North American Network Operators Group) meeting in Vancouver CBC engineers gave a great presentation on how they use Akamai and other CDNs (Content Distribution Networks) to deliver CBC TV and Radio content over the Internet in Canada and around the world. This type of delivery of CBC broadcast content is called OTT (Over The Top) is the same technique used by Netflix. OTT is critically important in Canada, especially for Canadian broadcast content and cultural material as we continue to see media consolidation in Canada (re Bell takeover of Astral media). The larger commercial telcos and cablecos hate OTT, have little interest in supporting Canadian broadcast content, other than that they are required to carry by regulation. When we eventually relax our foreign ownership restrictions on telecom and cable, there will be greater push by telcos and cablecos to be relieved of all Canadian content restrictions. OTT may be the only way we can insure that a Canadian voice will be heard in the future multi-media cacophony of competing services delivered over the Internet.IXPs as being developed by the Canadian Internet Registration Authority (CIRA) and a number of regional networks will be critical for delivering Canadian content OTT via CIRA’s integrated of Akamai with the IXP. Smaller ISPs and community networks who have a vested interested in promoting Canadian values and content need to distinguish themselves from the oligopolistic telcos and cablecos . CIRA’s leadership in deploying IXPs across Canada employing Akamai CDN will be critical for [...]

NSF and XSEDE survey on cloud use cases for researchers and educators


[It is good to see the National Science Foundation (NSF) and XSEDE (eXtreme Science and Engineering Discovery Environment) undertake a survey to determine cloud use cases by researchers and educators and plan accordingly for the seamless integration of cloud resources into the XSEDE architecture.
A good example of such a possible seamless architecture is SURFconext which uses university credentials via SAML for access to commercial clouds such as GreenQloud, etc. This is the kind of study that myself and co-author Dr Denis Therien recommended in a report we wrote for Canadian Foundation of Innovation on the future of cyber-infrastructure in Canada. In our report we uncovered considerable anecdotal evidence from researchers and from funding councils in Canada, USA and Europe that many researchers and educators are already using commercial clouds, paid out of their own pockets.

We also noted that of the many small and medium size research teams are acquiring their own clusters, they could instead be best served by commercial clouds. In our report we speculated that the total aggregate spend on these small clusters could possibly be greater than all the money spent on HPC. Unfortunately there is no way of tracking these expenditures as the purchase of small clusters is often buried amongst other larger equipment and research costs.

Most of these research teams are not your traditional compute intensive disciplines, and have little concern about their computer being in the top 500, and are for the most part focused on “occasional” computational data analysis. These teams are largely in the humanities, health sciences, biology sciences, civil engineering, etc. They refer to acquire their own clusters because it is far less hassle than applying for permission to use a large campus HPC facility. While a fully loaded university private cloud may be cheaper than commercial facilities, most small and medium research teams only need occasional use of such facilities and so often a commercial cloud is more convenient. As well, we also noted anecdotal evidence that a lot of the necessary tools and applications for many research teams are only available on commercial clouds. Many graduate students and small businesses are motivated to build tools for commercial clouds as they see a significant revenue opportunity.

I hope that the NSF and XSEDE will also undertake a more proactive analysis beyond a volunteer survey. Traffic to commercial clouds from R&E networks at major peering is sky rocketing. Tracking some of the IP addresses to determine who are the heaviest users of commercial clouds at universities may be more revealing than depending on a volunteers, especially outside of engineering and the physical sciences to complete a survey. – BSA]

The Future of R&E networks and cyber-infrastructure (eInfrastructure)


[A couple days ago I had the honor of giving a talk on the future of R&E networks and cyber-infrastructure to celebrate Kees Neggers receiving the Order of the Orange Nassau ( the Dutch equivalent of the Order of the British Empire). As many readers of this blog know Kees Neggers is retiring from SURFnet – the Dutch R&E network.  The following text is some excerpts from my talk. The full presentation is available at: discussing the possible future direction of R&E networks and cyber-infrastructure (einfrastructure) it is important to look back to see the significant impact that R&E networks have already had on the global economy as well as supporting research and education.  As many people know the Internet started with the R&E network community, beginning with the NSFnet in the US and quickly followed by many other R&E networks around the world including SURFnet.  The web, Internet browsers and many other critical tools were developed by this community. Almost all of the major Internet applications we know of today such as Google, Facebook, Twitter, LinkedIn, etc were first developed at university dormitories and laboratories by students who had access to these high speed research networks.The unfettered bandwidth and “permission free” environment made possible by universities connected through R&E networks enabled these students to create exciting new applications and services that would not be possible on commercial networks of the day. SURFnet, in particular, founded the world’s largest Internet Exchange point AMS-IX which has made The Netherlands a global hub for Internet networks and data centers. It also pioneered concepts in customer owned dark fiber and optical networks that has dramatically reduced the cost of broadband which in turn has enabled The Netherlands to become one of the world’s most advanced broadband societies.The direct and indirect economic impact of R&E networks in the development of the Internet and all these associated applications and services is worth trillions of dollars and represents at least 6% of our collective GNP.  This is something governments and funding bodies need to remember when deciding what initiatives to support in terms of innovation and creating economic wealth. Empowering our students at our colleges and universities with access to advanced Internet R&E networks will eventually create the next generation of entrepreneurs to bring forth innovative applications and services, resulting in new jobs and businesses.For example, Lev Gonick, CIO for Case Western university , recently noted that R&E networks are morphing into “entertainment” networks as the bulk of the IP traffic (over 60%) at many universities  is video streaming services such as Netflix, Hulu, video file sharing etc.  This is consistent with other data I have seen over the years, that the bulk of most IP traffic on R&E networks is destined for residences and dormitories, of which a substantial is entertainment or game based traffic. (Lightpath traffic is generally much more research intensive). As Lev pointed out this preponderance of social networking and entertainment traffic on university R&E networks is not a bad thing. Students are the leading adopters of advanced technology and when they are given the  freedom of having virtually unfettered bandwidth and few restrictions they can be very creative. New services such as R&E CDN networks, collaborative platforms, integrated wireless services, etc promise to leverage this entertainment aspect of R&E networks to facilitate a similar revolution as students in residences get exposed to these technologies and adopt them into new products and services out into the working world. Although R&E networks have already had a huge economic and societal impact, I think the[...]

How To Use Cloud Computing To Do Astronomy (and other sciences)


[Here is an excellent balanced presentation on the advantages (and disadvantages) of using commercial clouds for astronomy and other research programs.
 I have long argued that commercial clouds are not yet well suited for HPC applications, but they can play a vital role in helping medium and small science (“boutique”) science teams in addressing many of the mundane tasks  in handling their data deluge. See my recent CIFAR presentation

 Simple things like file transfer can be a major hurdle for researchers who are not computational scientists. Tools like  Globus on Line or SURFconext using commercial clouds are ideal for these boutique science teams in  simplifying or eliminating many of these mundane tasks. Use of these tools does not necessarily mean that the actual research computation is done in the cloud.  HPC facilities may still be needed to do the raw number crunching.

At a recent cyber-infrastructure event hosted by the Canadian Institute for Advanced Research (CIFAR) Dr Larry Smarr, chair of the National Science Foundation Cyber-infrastructure Committee,  pointed out that cyber-infrastructure is not a recent phenomenon.  A commitment to cyber-infrastructure reflects a long history in the US and other jurisdiction of the recognition of the importance of computation and networks to advanced science and commercial spin offs.  As Dr Smarr stated “ Cyber-infrastructure is not an option for advanced societies”.

Commercial clouds are going to play an increasingly critical component of cyber-infrastructure.  Although the authors of this study point out that the cost of commercial clouds, in some cases, can be more expensive than deploying your own cluster or HPC facility, even if you take into account depreciation and energy costs,  the big advantage of commercial clouds is “time to market”. While, in some cases a fully loaded HPC facility is cheaper, the time to get funding approval and then peer review to actually use the facility can take years.  With a commercial cloud a researcher can start immediately focusing on their science and scale up their application once they have sorted out the initial bugs and code. More importantly many graduate students and  researchers are moving their tools sets to the cloud for easy access by other members of their community, as well as for the potential to make money from various “click compute” initiatives.

While Canada may be far behind other nations in terms of developing a national cyber-infrastructure strategy with or without commercial clouds, being late has one advantage in that we can learn from other’s mistakes.  As well, rather than reinventing the wheel and trying to develop our own common science platforms or middleware, we can beg, borrow or steal from others. Most cyber-infrastructure middleware is open source and have many excellent examples we can use in Canada such as Globus On Line, COmanage, HubZero, SURFconext, NECTAR, etc – BSA]

How To Use Cloud Computing To Do Astronomy

R&E Network and Green Internet Consultant.

twitter:  BillStArnaud
skype:    Pocketpro

Why CDNs are critical to future of R&E networks, Big Data and the Internet


[Today Netflix announced that they are deploying their own Content Delivery Network (CDN) for delivery of their video streams to Internet Exchange Points (IXPs) around the world. More importantly they are making the hardware and software design of their CDN servers freely available. That means any network can deploy Netflix CDN boxes deep into their network to significantly reduce traffic volumes and improve performance for users. In addition to the Netflix announcement the IETF has started up a working group called CDNi which is looking at developing standards for interconnection and distribution of CDN networks globally.These initiatives will have a significant impact for R&E networks in terms of Big Data, ensuring the Internet remains open and for creating new revenue opportunities. It is not only movies and commercial web sites that benefit from CDN networks. Any large data set that requires wide distribution, especially to mobile wireless devices can benefit from a CDN network. The high energy physics LHCONE network is a good example of a CDN network designed for a specific big data application. But there are many other large data sets in genomics, astronomy, social sciences, etc that could benefit from a generalized R&E CDN facility. Researchers and educators, like everybody else, want access to their data any time, any place and on any device. CDNs are critical to realizing such a vision.To date CDN facilities have not been critical for R&E networks because of the ample bandwidth, but as more and more users are accessing the R&E networks through wireless connection, or through the commercial Internet (i.e. for Citizen Science or courseware applications), performance and throughput can be significantly enhanced with a CDN network. It is not only receiving content and data that CDN networks are important, but also for delivering content to the global Internet community. Unfortunately most commercial CDN networks do not carry research data or any type of public content such as courseware, public service multimedia, etc. That is why it is important that R&E networks deploy their own CDN networks, and like other CDNs deliver this content to commercial ISPs at IXPs and other facilities. In countries like Canada delivering content from small Canadian multimedia businesses and other organizations to fellow Canadians and the global community is also an important role for R&E CDN networks. Deploying a CDN network could also be a revenue opportunity for R&E networks in delivering content to commercial ISPs and community networks at IXPs on behalf of public broadcasters, museums, and other public entities. Public broadcasters such as PBS, CBC, TVO, BBC, etc are seriously looking to looking at using OTT (Over The Top) distribution networks (e.g. Netflix) for their future direction. R&E networks could significantly reduce costs for these public broadcasters (and yet still earn significant revenue for the R&E network) in delivering this public content to the global community. Working in partnership with community network initiatives, such as UCAN, Gig.U and public supported IXPs could be mutually beneficial for both R&E CDN networks and IXPs. A good example, as I mentioned in a previous blog the Canadian Internet Registration Authority (CIRA) is working with regional R&E networks to help deploy community IXPs with integrated support for multiple CDN suppliers. BCnet is another example which has deployed IXPs in small communities and is now looking at deploying CDN services to these IXPs as well.NORDUnet and AARNet are also well positioned to be global players in deploying public CDN networks and insuring the communities they serve have a global voice for their content. Both networks have major peering connections at a number [...]

Federated R&E networks take a step forward in Europe


[At the recent highly acclaimed Terena networking conference in Iceland there were several significant steps forward to move to “federated” R&E networks in Europe.Federated R&E networks is considered by many to be the new Internet architecture not only for the R&E community, but for the Internet as a whole. With federated networks there is far less hierarchical structure of campus, regional, national and pan national networks. Instead universities, regional networks and even individual university departments establish their own network connections to Open Internet Exchange Points (OIXs) and more traditional IXs and peer directly with each other.  These networks can then  interconnect to commercial Clouds and Content Deliver Networks (CDNs) as well wireless partners at the OIX and/or IX.  Some national and pan-national R&E networks still see federated networks as a threat to their existence as local institutions or regional networks can bypass their backbone and thereby undermines their current business model. But the role of R&E networks is not to insure their permanent existence, but instead make sure that the needs of the research and education community are addressed first and foremost, even if that means surrendering their traditional role as national aggregators.  Forward looking national and pan-national R&E networks have started to realize that this is the future direction for their network architecture and are now focusing on Net+ services in terms of their core service delivery.I am pleased to see that both Internet 2 with OpenFlow and now GEANT in their recent partnership with OpenNaaS have taken a step forward in the direction of supporting federated networks. ESnet as well is doing some very interesting work with OpenFLow in the last mile of regional networks.  OpenNaaS allows institutions or regional networks to create their own virtual IP network. It was built upon the foundations of Canada’s User Controlled LightPaths (UCLP) and the concept of Articulated Private Networks (APNs).  The original proposition of UCLP and now OpenNaaS is to allow end users or institutions construct their own networks with their own independent forwarding, management and control planes. These end user controlled networks interconnect with each other at OIXs.  Sadly in Canada, UCLP development has largely ground to a halt as the major development centers for UCLP and Software Defined Networks (SDN) - Communications Research Center and CANARIE have largely discontinued further work in these areas.  UCLP is also the foundation for the GreenStar network.There still remains many outstanding issues with respect to the deployment of federated networks in terms of issues that all regional networks or institutions have direct access to an OIX and need a backhaul facility. Who pays for these circuits and how they are managed remains a significant issue.  Policy and Governance of OIXs are still being debated in various forums as for example GLIF.  – BSA]Further readingGLIF paper on Open Internet Exchanges and OpenNaaS announcement OIX in London POPs in Europe Openflow for last mile end to end networking Network and Green Internet Consultant. email:     Bill.St.Arnaud@gmail.comtwitter:  BillStArnaudblog:    Pocketpro[...]

How Internet 2 is helping researchers make effective use of commercial clouds


[Here is an excellent article in HPC in the cloud on how the Internet 2 network is helping researchers and educators use commercial clouds.  Along with Ian Foster I have long argued that commercial clouds are ideal for small and medium science teams especially  in humanities, social sciences, bio-informatics, genomics, etc. Small and medium sciences teams, made up of one or two PIs, a couple post docs, technician and some graduate students is how the  overwhelming type of research is done at our universities.   They don’t have the IT human resources to manage large scale cyber-infrastructure facilities like physics or engineering, and in many cases they only need computing resources on an infrequent basis.  As such, in the past, they often purchased a small cluster that was often lightly utilized.  Many of the tools they need are now available on commercial clouds as small businesses and graduate students prefer to develop tools on such facilities because of the commercial revenue from click-compute business models. Over the past couple of days the Canadian Institute for Advanced Research – CIFR organized a cyber-infrastructure consultation meeting with senior Canadian researchers, funding councils and government departments. In the discussions at the meeting I was struck, by how simple tasks such as moving large data files is still a significant problem for most researchers, especially those outside of physics and computer science. Many researchers are still using FedX or snail mail to ship their data to fellow researchers. This is where Ian Foster’s work with Globus On Line team is so important – to develop tools using commercial clouds that eliminate or remove the mundane tasks that consume an inordinate amount of a researcher’s time such as file transfer, indexing, massaging data, cataloging etc etc. This does not mean, in many cases, that the researchers data or computation is done on the commercial cloud, with all the attendant problems of privacy and security, etcTo my mind making life easier for researcher should be the number priority for cyber-infrastructure. Many networks like JANET, SURFnet, NORDUnet are undertaking similar initiatives as Internet 2’s NET + services. – BSA]Ian Foster’s presentations on Research IT as a Service Services Satisfy a Higher Calling Cloud computing is enabling services at scale for everyone, from scientific organizations and commercial providers to individual consumers. Higher education, in particular, has many collaborative projects that lend themselves to cloud services, however often those services are not tailored to the uniqueness of an academic environment. For example, there are very few businesses that have their research department work with their competitors, whereas in higher education, most research educations occur between institutions. That's where the Internet2 NET+ project comes in. During their annual member meeting, the networking consortium announced the addition of 16 cloud services to its NET+ program, aimed at reducing the barriers to research. HPC in the Cloud spoke with Shel Waggener, Senior Vice President of Internet2 Net+, and Associate Vice Chancellor & CIO for University of California, Berkeley, to get the full story.Internet2 sees itself as a bridge between the academic communities and commercial vendors. "We're focused on cloud computing enabling scale for a community," Waggener stated, adding, "The ability to have any researcher, any student, anywhere at any institution and instantly use services together is a very powerful opportunity."Internet2 is probably best known for its 100 Gigabit E[...]

IXPs and CDNs critical to the future of competitive broadband Internet


[We continue to see consolidation in the broadband market and various games played by the cablecos and telcos to thwart competition or undermine network neutrality (See below).
Until regulators create true structural separation between infrastructure and service providers the chances of seeing genuine broadband competition are slim. It is interesting to note telecom regulators in North America have imposed structural separation in the past. In the 1970s when the cable industry was a fledgling startup industry the FCC in the US and the CRTC in Canada passed regulations forbidding telephone companies to acquire and/or compete with cable companies. This enabled the creation of a entirely new business sector – cable television- who now dominates the broadcast and Internet market place. If regulators and governments are interested in stimulating the economy and creating new business opportunities, it is time they study their past successes and breakup up today’s oligopolies by imposing structural separation and allow a true competitive market in broadband Internet.

In the mean time the one bright spot in the competitive marketplace is the development of Internet Exchange Points (IXPs) and the collocation of Content Distribution Networks (CDNs). In a recent a talk at RIPE-64 given by Kurtis Lindqvist demonstrated that more IXPs will be even more important as broadband speeds increase. With larger and larger data flows the need to interconnect at an IXP to a CDN network or peering network will becoming increasingly important. See
I am very pleased to see that Canadian Internet Registration Authority (CIRA) has taken a very important leadership role in Canada in this regard. (Full disclosure: I am a member of the CIRA board). CIRA has undertaken an active program to help qualified communities, independent ISPs, regional R&E networks and others to deploy IXPs in their community. CIRA’s overall goal is to have local members build and operate the IXP, with CIRA bringing technical expertise, stability, back office functions, governance assistance, content providers and, if required, some financial and gear support. Most significantly CIRA will help the IXP provide a variety of DNS hosting services (which can improve responsiveness and reliability for connected users) as well arranging CDN networks to collocate at the facility.

The combination of these services- peering, DNS and CDN will provide connected independent ISPs, R&E networks, community broadband networks and other organizations the capability to provide services to their targeted communities and provide a modicum of competition to the local incumbent oligopoly. This service by CIRA will be especially important for small business, community and R&E networks as they look to deliver or use cloud services and wireless applications to their local communities. The integration of WiFi with 3G/4G with anytime, anywhere, any device communications for education and research will also be critically dependent on these facilities.

Further reading:

7 ways Comcast is killing the cable killers

Keeping the Internet Neutral

R&E Network and Green Internet Consultant.
twitter: BillStArnaud
skype: Pocketpro

Software Defined Networks and integration of Wifi with 3G/4G for R&E networks


[A number of R&E networks such as SURFnet, JANET, AARnet, etc are actively promoting mobile services and looking at integration of campus Wifi with 3G/4G networks using Eduroam. Mobile wireless services promises to be major service offering for R&E networks as the Internet of Things and Machine to Machine (M2M) becomes increasingly critical for research. Applications such as personal medical devices on (or in) the body, environmental sensors, traffic monitors and even garbage truck tracking will need such networks. As well anytime, anyplace, any device education and research will be increasingly dependent on the integration of campus Wifi, community Wifi and 3G/4G networks. Public content and distribution networks will also be an integral component. And as I have blogged in the past such wireless integration allows the deployment of overlapping Green WiFi nodes – powered by solar panels which will be needed to adapt a warmer climate. Here is a great article on OpenRadio a project from Stanford that hopes to use OpenFlow to create pools of available broadband from Wi-Fi, cellular and other networks. The project team is working with Texas Instruments to build $300-$500 base stations for the hardware component, while researchers try to build the orchestration software. Hopefully the base stations can be powered by renewable energy. R&E networks and campus IT staff could direct al bandwidth hungry applications to their WiFi networks while using much more expensive 3G/4G for e-mail and text messaging.—BSA][..]By layering the orchestration software on top of the networks, operators can easily write programs that can help them optimize their networks. For example, an operator could limit Netflix or YouTube traffic to only 40 percent of the LTE airwaves and save the remainder for other data traffic and voice.Right now, operators have to buy expensive gear and make tweaks across their entire network to allocate their bandwidth for certain services. OpenFlow makes the network programmable and easy to tweak using higher-level programming languages. Katti says that by using programs to manage the flow of traffic across a pool of network resources, operators could alleviate the so-valled “spectrum crisis.” From a consumer perspective moving form a Wi-Fi to a cellular network would become seamless under the OpenRadio vision.Katti’s ideas are compelling, especially for less traditional operators such as Republic Wireless or Free in France. Both operators offer mobile phone service that rely primarily on the Wi-Fi networks around a user and use the 3G networks as a last resort. Given the right hardware and the OpenRadio software they could make managing their networks easier for them and for their users. [..]But for carriers, while this might address their spectrum worries it also is a threat to their business model, which is built around perceived scarcity. Verizon held off on including Wi-Fi in its phones for so long because it wanted to shunt consumers to its cellular network, where the costs per gigabyte of data used are higher. If OpenRadio takes off, it’s easy to envision companies trying to buy service from a wholesaler (maybe Sprint will step up) to create wireless networks out of Wi-Fi, white spaces or other airwaves. Enterprising carriers or hot spot operators might even set up roaming agreements that make such coverage global. I’d love to see OpenRadio make it out of Stanford into the real world. ------R&E Network and Green Internet Consultant. email: Bill.St.Arnaud@gmail.comtwitter: BillStArnaudblog: Pocketpro[...]

European R&E networks- cloud development and Green IT


[Here are 2 good sources on the latest developments with European R&E networks on providing cloud services to their communities and Green IT.
A lot of focus on the clouds and networks in Europe is on their potential energy and carbon reduction. For the most part they are a very good summary of the issues, but I still remain concerned that most R&E networks remain too focused on mitigation rather than adaptation. We seriously need to think of the consequences of a much warmer planet and severe weather patterns. But as well we need to be conscious of a growing protest movement who are threatening to block coal trains and shut down coal plants. What will happen to your network, your cloud and your institution if there are long periods of rolling blackouts or brownouts and ultimately a time when governments order the shut down of all coal plants? Someday soon the public will wake up and realize coal plants are a lot worse than nuclear plants in terms of their environmental impact. If countries are willing to shut down their entire nuclear industry because of a very small perceived threat, it is only time before they realize we need to shut down coal.
When there is no power, energy efficiency is irrelevant. How do you build and run a network or cloud when there is little or no power from the grid? How do you provide network services to clients when they have no power? These are the questions we should be asking in terms of the design of future networks and clouds.
Another minor complaint I have is comparing cost of commercial cloud core versus that of a fully utilized core on an academic cloud. There is no question that the commercial cloud can be twice as expensive as academic cloud – even if you don’t take into account amortization expenses. But the huge advantage of commercial clouds to funding agencies is that there is no large up front expense to use the facility. Money that otherwise would be tied up in expensive hardware can be better utilized in supporting other research – BSA]

Terena Green Workshop

Green GÉANT Team advancing environmental initiatives
Case study: GEANT and GreenStar Network

CSC Building a Green Data Center in North

JISC-SURFnet-CSC workshop on clouds
R&E Network and Green Internet Consultant.
twitter: BillStArnaud
skype: Pocketpro

7 reasons why R&E networks and Universities are critical to future of broadband


[There has been considerable discussion about the future of broadband in terms of infrastructure i.e. fiber, wireless, community owned etc . However, there has been little discussion, to borrow a phrase from Internet 2 , on Net+ broadband services. It is in the Net+ services where I think R&E networks can play a critical in helping communities and small commercial ISPs deploy advanced services and applications that will provide new business models to underwrite the costs of next generation broadband. The next generation broadband I believe will look like architecture of existing R&E networks, rather than the monolithic walled gardens of the telcos/cablecos. Partnering in next generation broadband is also in line with the core missions of R&E networks, schools, libraries and universities in terms of the future of data intensive researcher from thousands of distributed sensors and delivering research and education to any device, any time, anyplace. The fundamental factor driving a new vision of next generation broadband is the fact that Net+ services such as clouds and content distribution are localizing traffic (i.e over 90% of Internet traffic will appear to be locally sourced – even though actual sites they may be accessing sites 1000s kms away). I believe through the widespread application of Net+ services will also create a whole new innovation and economic eco-system much in the same way R&E networks enabled the original Internet. See my paper on “Personal Perspective on Future of R&E networks” A great example of this thinking is the recent announcement of ESPN (the major US sports network) to partner with Internet 2. RT See also how clouds and Net+ services are enabling an entire new innovation ecosystem and thousands of new startups 7 important areas where R&E networks can play an important role in advancing and support community broadband are as follows:1. Encouraging universities, colleges, schools and libraries to be anchor institutions in a distributed broadband network architecture. This does NOT mean that R&E networks will provide basic Internet to homes or commercial enterprises, but these anchor institutions and R&E networks can host a number of Net+ services to interconnect to services providers, critical to the community such as distributed content caching, integrated 4G/Wifi nodes and local peering. This will also allow these institutions to deliver their services to the community via any device, anyplace any time. See New OTI Whitepaper, "Universities and R&E networks as Hubs for Next-Generation Networks": Developing Open Content Distribution Networks to integrate Net+ cloud and content services into one seamless service offering. See” Why R&E Networks Should Be Aware of the CDN Interconnect Initiative (CDNI)” . See also Google and Akamai develop new caching technology and protocols to speed up Internet over 3G/4G networks Building enterprise centric integrated WifI/4G networks, versus telco/cableco attempt to make public Wifi become part of walled garden cell phone network. See Wi-Fi offloading: Who controls your handset? See JANET, AARnet and SURFnet initiatives Building community IXPs or TXPs, local peering and open collaborative exchanges (OXPs). See great presentation a[...]

Great example of public-private network-computer partnership to support big data research


[Here is a great example of how optical exchange points and advanced networks (STAR LIGHT) working in partnership with public clouds can accelerate fundamental research. Such partnerships can significantly reduce capital cost of campus computing resources as well as operational costs in terms of energy consumption of on campus computing. Thanks to Ed Lucente for this pointer – BSA]Open Cloud Consortium Announces First Integrated Set of Cloud Services for Researchers Working with Big Data________________________________________CHICAGO, April 4 — Today, the Open Cloud Consortium (OCC) announced the availability of Tukey, which is an innovative integrated set of cloud services designed specifically to enable scientific researchers to manage, analyze and make discoveries with big data.Several public cloud service providers provide resources for individual scientists and small research groups, and large research groups can build their own dedicated infrastructure for big data. However,currently, there is no cloud service provider that is focused on providing services to projects that must work with big data, but are not large enough to build their own dedicated clouds.Tukey is the first set of integrated cloud services to fill this niche.Tukey was developed by the Open Cloud Consortium, a not-for-profit multi-organizational partnership. Many scientific projects are more comfortable hosting their data with a not-for-profit organization than with a commercial cloud service provider.Cloud Service Providers (CSP) that are focused on meeting the needs of the research community are beginning to be called Science Cloud Service Providers or Sci CSPs (pronounced psi-sip). Cloud Service Providers serving the scientific community must support the long term archiving of data, large data flows so that large datasets can be easily imported and exported, parallel processing frameworks for analyzing large datasets, and high end computing."The Open Cloud Consortium is one of the first examples of an innovative resource that is being called a Science Cloud Service Provider or Sci CSP," says Robert Grossman, Director of the Open Cloud Consortium. "Tukey makes it easy for scientific research projects to manage, analyze and share big data, something this is quite difficult to do with the services from commercial Cloud Service Providers."The beta version of Tukey is being used by several research projects, including: the Matsu Project, which hosts over two years of data from NASA's EO-1 satellite; Bionimbus, which is a system for managing, analyzing, and sharing large genomic datasets; and bookworm, which is an applications that extracts patterns from large collections of books.The services include: hosting large public scientific datasets; standard installations of the open source OpenStack and Eucalyptus systems, which provide instant on demand computing infrastructure; standard installations of the open source Hadoop system, which is the most popular platform for processing big data; standard installations of UDT, which is a protocol for transporting large datasets; and a variety of domain specific applications.Tukey has a direct 10 Gbps connection to StarLight, an advanced national and international communications exchange facility, which in turn connects to dozens of high performance research networks around the nation and the globe. "Tukey enables scientists to share their big datasets with researchers around the country and the world," says Joe Mambretti, Director, International Center for Advanced Internet Research (iCAIR) at Northwestern University.About the Open Cloud ConsortiumThe Open Cloud Consortium (OC[...]

Network as a Service (OpenNaaS) - software defined networks taken to new level


[It is exciting to see the Mantychore team in Europe undertake to move software defined networks to a new level of sophistication and ease of use.
Mantychore is currently producing a tool to automate the generation of IP networks within the framework of Network as a Service called: “OpenNaaS.” OpenNaaS is built on some earlier concepts developed in the CANARIE UCLP program, now extended to IP networks and low carbon architectures like Greenstar.

The Mantychore team will be hosting a workshop at the Terena meeting in May. During the workshop there will be active engagement with any potential OpenNaaS users to meet their business needs with an emphasis on practical implementations. There will be demonstrations of OpenNaaS and discussion on how communications industry and service providers can leverage Mantychore Open NaaS to develop new business models—BSA]

Mantychore OpenNaaS

Mantychore Open NaaS in the Greenstar network
R&E Network and Green Internet Consultant.
twitter: BillStArnaud
skype: Pocketpro

European study (Terena) on the Future Role of R&E Networks and financial sustainability - ASPIRE.


[I am pleased to see Europe, under the auspices of Terena is undertaking an in depth study of the future role of research and education networks. To my mind R&E networks will continue to play a critical role not only in serving the needs of the research and education community, but in also in defining new innovative services as well as Internet and broadband business models that don’t require incumbent monopoly solutions. It was the R&E networks that introduced many innovations including the Internet itself, but also the web and infrastructure innovation like customer owned dark fiber, condominium fiber, optical Internet exchange points, software defined networks, green IT, wireless roaming etc.However, in the age of global financial constraint R&E networks have to face reality. With nearly bankrupt governments, aging population and health care consuming more public dollars there is going to be less money in the coming years for higher-ed and networking. No matter how great a job they are doing R&E networks are going to have to expect less and less money from government. Most R&E networks today are operationally self sufficient from membership fees etc, and only look to government for financing of innovation and/or capital projects. . Some innovative networks like AARnet, Internet 2, NORDunet, SURFnet etc already substantially independent of government funding, except for some small programs for innovation, etc. In future all networks are going to increasingly have to look to their connected institutions to underwrite a substantial portion of the network costs. As you can see from the Terena ASPIRE study, many forward looking R&E networks are already developing strategies to deal with a future of little or no government funding.Unfortunately, most education and research institutions are also under financial stress and cannot forward higher fees. So R&E networks are going to have to find innovative solutions that not only reduce costs for their members, but also provide new revenue opportunities for the network itself. Sometimes financial necessity can be the mother of invention. A good example is content networking and peering. This dramatically reduces Internet costs for connected institutions and enables a new business model that avoids the old mindset of dollars per megabyte. NORDUnet, AARnet and Internet2 has been exemplary leaders in this regard. If all global R&E networks worked together they could represent themselves as a global Tier 1 network and virtually eliminate transit fees through a content and peering strategy.R&E networks are going to have work closely with their connected institutions to develop coordinate cyber-infrastructure solutions. Two institutions that are great example of this strategy is Cal_IT2 and Indiana U (it helps that the president was a former CIO). But R&E networks and governments can also help promote these developments by supporting energy and green revolving funds to underwrite many of the costs of developing commercial cloud and cyber-infrastructure solutions. UK government and JISC are doing some innovative work in this area of promoting use of revolving funds.Brokered commercial clouds, outsourcing campus IT, mobile services, “research as a service”, collaborative eScience platforms, green IT, industrial incubation, community anchor institutions, deploying next generation broadband and broadband transit exchanges are other innovation examples that have potential to generate revenue for R&E networks. The ASPIRE team will be looking [...]

Critical role for R&E networks+commercial clouds in US government Big Data initiative


[It is great to see US and European governments undertake initiatives to promote the development of research into Big Data utilizing commercial clouds. Many cloud providers are offering free resources to support these initiatives. R&E networks will play a critical role in linking researchers to the commercial clouds and developing collaboration platforms and portals. The recent Apache-Rave announcement in partnership with XSEDE and COmanage in the US and SURFconext in Netherlands is a great example of developing “Research as a Service” using commercial clouds . See Ian Foster presentation. Peering with commercial cloud providers will also be critical. I have long argued that development of commercial clouds to support research will fundamentally change cyber-infrastructure at universities. As Dr Ed Lazowska commented in a New York Times article: “The need to analyze vast amounts of data from a broad array of sensors is going to be far more pervasive than the use of numerical simulation - even though the use of numerical simulation continues to increase. Even in fields such as national security and scientific discovery, for decades the flagships for HPC, large-scale data analysis is growing to equal importance. And this requires entirely different hardware and software architectures than does traditional HPC. “ HPC will remain an important niche, but analyzing large volumes of data is ideally suited for commercial clouds.Many people have argued for public funded academic clouds. The big disadvantage of an academic cloud is that it requires new infrastructure updates every few years in order to meet ongoing demand for additional computation resources. So the situation, from a funding council perspective is an ongoing requirement to continuously upgrade computer resources whether they stand alone systems or are lumped together within an academic cloud. But with commercial clouds funding agencies do not have to purchase infrastructure to enable researchers to use these facilities. Commercial clouds make the necessary investment to upgrade their infrastructure over time as demand warrants. Many commercial cloud providers spend hundreds of millions per year on computer upgrades – which dwarfs the annual expenditure most funding councils spend on HPC facilities.Many R&E networks are providing brokered commercial cloud services which will further reduce cost of using clouds (for those that are not free) – BSA]OBAMA ADMINISTRATION UNVEILS “BIG DATA” INITIATIVE:ANNOUNCES $200 MILLION IN NEW R&D INVESTMENTS to make the most of the fast-growing volume of digital data, the Obama Administration today announced a “Big Data Research and Development Initiative.” By improving our ability to extract knowledge and insights from large and complex collections of digital data, the initiative promises to help solve some the Nation’s most pressing challenges.To launch the initiative, six Federal departments and agencies today announced more than $200 million in new commitments that, together, promise to greatly improve the tools and techniques needed to access, organize, and glean discoveries from huge volumes of digital data.[…]National Institutes of Health – 1000 Genomes Project Data Available on Cloud: The National Institutes of Health is announcing that the world’s largest set of data on human genetic variation – produced by the international 1000 Genomes Project – is now freely available on the Amazon Web Services (AWS) cloud. At[...]

Why R&E networks should be aware of the CDN Interconnect initiative (CDNI)


At the recent IETF meeting there has been considerable discussion about interconnection of Content Delivery Networks. A lot of this is being driven unfortunately by the incumbent telco/cableco’s who never understood CDN in the first place, and now want to assert control over this critical new Internet architecture, much in the same way that they want to take control over open WiFi hot spots as part of an integration strategy with their 3G/4G networks. R&E networks are the only independent organizations that have the knowledge and independence that can develop alternate strategies that don’t assume a “telco/cable uber alles” strategy. A good example is the Eduroam program which is now being used to seamlessly integrate WiFi with 3G/4G on networks like SURFnet, JANET, AARnet etc.CDNI will be critical to a future anywhere, anyplace, anytime education and research strategy. It will also be critical to those R&E networks that operate transit or internet exchange points for community or anchor institution networks. Most R&E networks have come to realize that CDN and peering is critical to their core business functions. On some networks over 90% of the traffic is CDN and peering. It enables most R&E networks to become self sufficient and yet provide a much lower cost value proposition to their connected institutions, eliminating the dollars per Megabyte mindset of the incumbents. – BSA]For more information on this topic please see:A personal perspective on the evolving Internet and Research and Education Networks report: Internet Traffic Exchange Points Distribution Network Interconnection (CDNI) Problem Statement volume of video and multimedia content delivered over theInternet is rapidly increasing and expected to continue doing so inthe future. In the face of this growth, Content Delivery Networks(CDNs) provide numerous benefits: reduced delivery cost for cacheablecontent, improved quality of experience for End Users and increasedrobustness of delivery. For these reasons CDNs are frequently usedfor large-scale content delivery. As a result, existing CDNProviders are scaling up their infrastructure and many NetworkService Providers (NSPs) are deploying their own CDNs.It is generally desirable that a given content item can be deliveredto an End User regardless of that End User's location or attachmentnetwork. However, a given CDN in charge of delivering a givencontent may not have a footprint that expands close enough to the EndUser's current location or attachment network, or may not have thenecessary resources, to realize the user experience and cost benefitthat a more distributed CDN infrastructure would allow. This is themotivation for interconnecting standalone CDNs so that theircollective CDN footprint and resources can be leveraged for the end-to-end delivery of content from Content Service Providers (CSPs) toEnd Users. As an example, a CSP could contract with an"authoritative" CDN Provider for the delivery of content and thatauthoritative CDN Provider could contract with one or more downstreamCDN Provider(s) to distribute and deliver some or all of the contenton behalf of the authoritative CDN Provider. The formation anddetails of any business relationships between a CSP and a CDNProvider and between one CDN Provider and another CDN[...]

Europe to deploy commercial clouds to support big science


[It is great to see Europe take an important leadership role and recognize the important role that commercial cloud providers can play in providing solutions for big science through their recent Helix Nebula – the Science Cloud announcement. It is estimated that 1/3 of the science applications running on expensive HPC facilities could easily run on commercial clouds, freeing up these facilities to focus on true high computational problems. As governments are under increasing financial pressure we need to find creative solution such as these types of partnerships to support big science in an era of massive data (See Ian Foster presentation below).To get around data privacy issues I hope European governments will also look at cloud franchise models as now being offered by Fujitsu (representing Microsoft Azure). With franchise models the physical infrastructure can be opened and operated by Europeans and therefore not subject to US Homeland Security rules and privacy regulations. The software and middleware can then still be consistent with global standards such as Amazon.Although details are still sketchy I hope the Europeans focus on developing middleware and cloud applications rather than funneling most of the money into infrastructure. As the Economist magazine pointed out the economic and business opportunities are in developing the cloud applications and middleware. Cloud infrastructure is a commodity business – best operated and funded by large commercial operators. Initiatives like JISC cloud applications program or XSEDE Globus On Line to relieve researchers from mundane tasks are good examples.Finally I believe the entire infrastructure could be paid for, in energy and carbon savings, if they ensure the cloud infrastructure is 100% zero carbon by locating the facilities at sites that use 100% renewable energy. – BSA]New pan-European cloud computing infrastructure will support scientific research how we provide science IT in an era of massive data but modest budgets Plan f or a Scientific Cloud Computing infrastructure for Europe #1 Establish a Cloud Computing Infrastructure for the European Research Area serving as a platform for innovation and evolution of the overall infrastructure. Goal #2 Identify and adopt suitable policies for trust, security and privacy on a European-level can be provided by the European Cloud Computing framework and infrastructure. Goal #3 Create a light-weight governance structure for the future European Scientific Cloud Computing Infrastructure that involves all the stakeholders and can evolve over time as the infrastructure, services and user-base grows. Goal #4 Define a funding scheme involving all the stake-holder groups (service suppliers, users, EC and national funding agencies) into a Public-Private-Partnership model to implement a Cloud Computing Infrastructure that delivers a sustainable and profitable business environment adhering to European-level policies.------R&E Network and Green Internet Consultant. email: Bill.St.Arnaud@gmail.comtwitter: BillStArnaudblog: Pocketpro[...]

Amazon's Simple Workflow and Software Defined Networks could transform cyber-infrastructure


[It is very exciting to see Amazon’s new simple Workflow service. Workflow, when combined with software defined networks, could transform the future of cyber-infrastructure or eInfrastructure.
Large scale instruments could forward their data sets and/or streams to Amazon for processing over specially configured networks designed to link together databases, Amazon web services, visualization engines and off site storage. A great example is how NASA is using Amazon’s Simple Workflow for processing images from the Mars landers. This is the fulfillment of the promise of UCLP – to build what we called “Articulated Private Networks” where every network element, computation, node, instrument was represented as a web service which could composed into a end to end network solution using BPEL workflows by the end user. This new Amazon service gives you the ability to build and run distributed, fault-tolerant applications with MapReduce and Hadoop that span multiple systems (cloud-based, on-premise, or both) which further enables deployment of zero carbon networks and clouds such as Greenstar or Hewlett-Packard GreenCloud. – BSA]

Amazon queues up new workflow service

Amazon’s Simple Workflow

NASA Case study using Amazing Simple Workflow

Articulated Private Networks with UCLP
R&E Network and Green Internet Consultant.
twitter: BillStArnaud
skype: Pocketpro

Are cloud applications blocking genuine HPC users from getting necessary resources?


[Over the past few weeks I have had several discussions with administrators from large HPC facilities in USA and Canada, who are part of the XSEDE and Compute Canada consortia respectively.
A common complaint I hear that is many of their high end resources are tied up supporting applications that are more ideally suited to operate on commercial clouds. Because many compute consortia like XSEDE and Compute Canada are largely supported through funding councils like NSF and CFI, researchers do not have to pay for access to these high end facilities. Virtually every researcher who makes an application through a peer review process is guaranteed a certain number of resources such as storage and/or computation on this infrastructure. But as a result many loosely coupled, highly parallelizable applications are consuming resources that would be much better suited for true HPC applications that are tightly coupled, compute and memory intensive. I have heard estimates of as much as 30-50% applications running on HPC facilities are in this loosely coupled category.

Many loose coupled applications could easily run on commercial clouds. Tightly coupled, compute intensive applications cannot run on clouds (at least not yet). Unfortunately researchers who have a loosely couple application don’t have an option of running them on commercial clouds because there is no funding program to support these initiatives. Their only option is to acquire their own computational resources (closet computing) or eat up precious, albeit free, resources available through compute consortia such as XSEDE and Compute Canada. True HPC researchers get short changed in the process. Compute consortia are reluctant to push certain applications to the cloud as this undermines their own justification to exist and possible future funding.

Anecdotal evidence is not data. I would be curious to know if anybody has statistics on the types of applications that run on their HPC facility? It would be interesting to get a true measure to see if applications suitable for the cloud are blocking genuine HPC users from getting the maximum benefit out of their machines. – BSA]

More companies deploying follow the wind/follow the sun technology


[It is exciting to see more companies embrace the concepts of software defined networks, network virtualization and follow the wind/follow the sun networks pioneered by CANARIE through its UCLP (Argia) and Greenstar network initiatives.
Big companies like Hewlett-Packard, AMD, Ericsson as well as many start-ups such as ConteXtream, Embrane, Big Switch, and now NIcria are now starting to deploy this technology. “NTT, which operates data centers around the world, uses Nicira’s software to move its desktop-as-a-service offering from data center to data center within Tokyo ahead of rolling brown outs in the wake of the Fukushima nuclear disaster. That’s right, we’re talking about cloudbursting — or moving a workloads on the fly from one data center to another.” The Nicria announcement also uses a concept originally developed by a research project called Mantychore in Europe where users can purchase virtual network resources as required, rather than paying for them upfront as a capital resource. These technologies will play an increasingly important future role in integrated WiFi/3G networks and enabling integrated remoted data centers as universities and business start to address the challenge of adapting to climate change – BSA]

Nicra annoucement

Use of Network Virtualization and Software Defined Networks to enable universities to adapt to climate change


Argia and High Performance Digital Media

Harmony UCLP Argia

Greenstar Network

Mantychore Project

Hewlett-Packard, AMD and Clarkson University Project
R&E Network and Green Internet Consultant.
twitter: BillStArnaud
skype: Pocketpro

JANET's innovative M2M 3G wireless service with Eduroam global roaming


[Many people may have read the excellent OECD report on machine to machine (M2M) communications which is expected to be the next big thing in terms of the Internet of Things. As noted in the Gigaom repot “Machine to machine networks, sometimes called the Internet of things, are the logical extension of today’s connected society, but creating such a network will require multiple technologies; telcos to open up their networks; governments to figure out a way to assign unique numbers for each device on the network; and new rules to protect security and privacy. In short, while the idea is fairly mature, the tools to make it a reality are lagging. To outline what still needs to be done, and give governments a framework for understanding how 50 billion devices could be connected in the next 8 years, the OECD has released a report laying out the needs of an M2M network and the tradeoffs associated with different technologies. “One of the big regulatory and technical challenges is for highly mobile devices like medical sensors attached to your body. If you are dependent on these devices for your research or more critically your health, it is not very reassuring to realize that carrier roaming agreements may make these devices inoperable or too expensive to use, outside of your carrier’s serving region.This is where R&E networks with their global Eduroam service can play a critical role. It will be decades before regulators assert rationale global roaming and data interchange agreements on the carriers. Despite their best efforts they have been unable to do this domestically. The OECD is naive in thinking that telcos will open their networks any time in the near future. Just as the R&E networks disrupted traditional old boy’s club of settlement based telecom with the introduction of the Internet, I believe the R&E networks have a critical role in doing an end run around the telcos to deliver a seamless, global wireless M2M service. A good example of such a strategy is the UK’s R&E wireless network service offering. Through JANET researchers and students can acquire 3G SIM cards for their cell phones or M2M devices with a variety of pricing plans and data rates. Right out of the box these devices support Eduroam authentication which means that these devices will work seamlessly with any other international 3G R&E wireless service that supports Eduroam.What is more interesting is JANET is negotiating arrangements with various suppliers like Greyhound bus to offer Eduroam authentication while on the move through areas of spotty or non existent 3G service. Integrating with national and international WiFi/3G networks like Starbucks and Google’s rumored networks using Ericsson/Bel Air technology is also conceivable. Next generation solar/wind powered Wifi/3G nodes will also allow direct optical wavelength interconnection into national R&E networks.JANET’s 3G M2M SIMs is a good pointer on how R&E optical networks can integrate with LTE/WiFi towers located at schools and universities. Radio- optical network backhauling one network will not rule them all[…]------R&E Network and Green Internet Consultant. emai[...]

NORDUnet's brilliant Internet peering strategy


[NORDUnet, the R&E network connecting the Nordic countries has recently undertaken a brilliant Internet peering strategy that will have global significant ramifications for supporting research and education around the world. NORDUnet is now emerging as one of the world’s first “GREN”s – Global Research and Education Network. NORDUnet is extending their  network infrastructure to multiple points of presence throughout the USA and Europe to interconnect to major Internet Exchange Points (IXPs). This will allow them to negotiate as a Tier 1 Internet service provider and exchange traffic with other global commercial Tier 1 Internet transit providers.  NORDUnet is also playing a global leadership role by extending this service offering, on a shared cost basis,  to NRENs such as SURFnet (Netherlands),  PIONIER (Poland) and perhaps others.Many network operators ask why they should build an extensive peering network when transit prices are only marginally more expensive than peering (and still dropping)? The NORDUnet engineering team are one of the first to understand that Internet peering is not about cost comparison between peering and transit pricing.Most universities (as well as consumers and business) have a fixed budget for Internet connectivity.  So regardless of traffic volumes they can only spend so much money for Internet transit.  As result many institutions cap traffic volumes to commercial transit providers.  But peering traffic is done on a settlement free basis and therefore traffic volumes are not linearly related to cost.  Many NRENs have discovered that content peering traffic has a huge benefit for their connected institutions in stabilizing costs without restricting use of the network.  On some NRENs, content peering traffic is now 90% of their overall traffic volume.  By connecting to the major IXPs in the USA, NORDUnet can eliminate purchase of virtually all  transit traffic.  Traffic volumes are expected to immediately jump because now institutions will not have to cap formerly transit traffic. This arrangement will have a huge benefit for the research community as more and more computational research is done on commercial clouds in the US.  NORDUnet realizes, that despite concerns about US Patriot Act, researchers are voting with their wallets and using commercial cloud providers and value added cloud providers in the US. Many research disciplines, especially genomics and bio-informatics are being increasingly dependent on commercial application providers, because they have the necessary tools critical to their research.  Numerous bioinformatics companies, like SoftGenetics, DNAStar, DNAnexus and NextBio, have sprung up to as they have found life sciences a fertile market for products that handle large amounts of information.  Access to these commercial organizations through the commercial Internet or Open Lightpath Exchanges is essential for the future of research.This initiative by NORDUNet will have profound implications for the future of the Internet and data intensive science.  The obvious next step after exchanging peering traffic is also to use this links for dynamic lightpaths and virtual networks for large data flows.  It is no surprise that networks like NORDUNet and SURFnet are also leading the developments of dynamic optical networking through GLIF.  [...]