By Alex Woodie
Cloudera today launched Cloudera Streaming Analytics (CSA), a new subset of the Cloudera DataFlow (CDF) platform that is based on the Apache Flink stream processing engine. The new software runs on existing Hadoop clusters and will be positioned to process large amounts of IoT and event data.
Since the merger with Hortonworks one year ago, Cloudera has taken more of an interest in managing streaming data, in addition to data at rest. As part of that effort, it has positioned the former Hortonworks Data Flow product (now CDF), as one of the primary product families next to the recently released Cloudera Data Platform (CDP) that offers the best of Hortonworks and Cloudera Hadoop distributions.
Cloudera has supported Apache Spark and its popular Spark Streaming engine on its Hadoop distribution for some time. It also supports Apache Storm and Kafka Streams. However, some in the community, including Hadoop co-creator Doug Cutting, consider Apache Flink’s dataflow architecture superior in some ways to other approaches, notably Spark Streaming, which originally used micro-batch techniques instead of continuous processing.
Flink certainly impressed Cloudera enough to include it in CSA. The company is positioning CSA running atop Hadoop as an end-to-end platform for a range of streaming use cases, from telco network monitoring and fraud detection to clickstream analysis and content recommendations, and it’s counting on Flink to deliver the goods.
Cloudera says it chose Flink primarily because of its scale, and its capability to process millions of data points and complex events in real time every day. Flink’s fault tolerance, data distribution, communications, and its ability to process data at rest also worked in its favor, according to a blog post written by Cloudera’s head of product marketing, Dinesh Chandrasekhar.
“While Cloudera offers our customers several options for stream processing engines – Storm, Spark Structured Streaming, and Kafka Streams, the addition of Flink to CDF is very significant,” Chandrasekhar wrote. “Storm has been slowly losing favor in the market and in the open-source community and users are looking for a better option to move into. Apache Flink is that option.”
The other frameworks have their own relevant use cases around stream processing and analytics, Chandrasekhar says. “However, Apache Flink has a streaming-first (over batch) approach to processing high-volume streams of data at high-scale, while supporting key features such as stateful streaming, exactly-once delivery, built-in fault tolerance/resilience, and advanced windowing techniques,” he says. “This makes it the default choice for a wider range of stream processing use cases.”
CSA is based on Flink version 1.9.1 and will install on CDP Data Center clusters running YARN. It will be able to read data from HDFS and Apache Kafka topics, and write to HDFS, Kafka, and HBase. CSA pipelines will be defined using Java DataStream and ProcessFunction APIs, and will utilize Cloudera Schema Register to manage the serialization and deserialization of events, according to Cloudera. It will support TLS, Kerberos, and exactly once processing semantics.
“Cloudera Streaming Analytics unlocks net new business value for businesses requiring real-time insights from fast-paced customer experiences,” writes Chandrasekhar. “Apache Flink provides stateful analytics at low latency and high scale to address such needs of today’s businesses.”
Apache Flink emerged from the Stratosphere research project at the Technical University of Berlin in 2009, and became a top-level project at the Apache Software Foundation in 2015. The in-memory framework was supported atop YARN from the beginning, but wasn’t restricted to running on Hadoop, which gave it certain advantages.
Development of Flink was spearheaded by the German company data Artisans, which launched a commercial version of Flink called the dA Platform in 2016. Before being acquired by Chinese Web giant Alibaba one year ago, the company changed its name to Ververica and added support for ACID transactions on streaming data with its Streaming Ledger offering.
Over the years, Flink has attracted a number of customers, including Disney, Netflix, and Uber, all of whom deployed Flink into production.
The post Apache Flink Powers Cloudera’s New Streaming Analytics Product appeared first on Datanami.
Read more here:: www.datanami.com/feed/
As we embark on a new year and decade, it seemed worthwhile to take a peek at the principal forums for global 5G industry technical collaboration and do a quick assessment of what is occurring and who are the “leaders.” The leadership dimension is especially relevant in Washington these days — which is suffering from a peculiar 5G dementia. As the year ended, there were no less than 35 current 5G related Congressional legislative actions, several of which actually passed one of the chambers. One is literally named “promoting United States international leadership in 5G.”
These efforts in an alternative reality are orchestrated by a recently appointed loyal White House drone with numerous adhoc K-Street lobbying organizations smoking the 5G dope produced around town. (Cannabis is now legal in Washington.)
5G Leadership Reality
There are many ways to measure leadership in the 5G arena, but one of the most compelling and measurable is to examine the participation in the principal industry technical venues where all the parties collaborate together in deciding 5G features and instantiating them in global specifications that everyone implements in products and services worldwide. Those parties consist of chipset vendors, equipment manufacturers, service providers, network operators, support organizations, institutes, and government agencies.
Participation is also a considerable undertaking because it requires the necessary resources to understand the subject matter and what is occurring across the dozens of bodies that are meeting now almost every month. The principal defining body for 5G — 3GPP — now has 1,114 work items (435 legacy, 601 full 5G, 78 next-generation 5G). The full “stand-alone” 5G specs are known as Release 16 and scheduled for adoption this year. The participation metrics of both attendance and substantive submissions are the ultimate reality test of who are the serious parties in the real 5G world. Less measurable are the “bonus points” for leadership acquired by initiating and leading particularly innovative new work and 5G features.
The meetings for advancing this work are hosted among the participants, and now typically occur every month — constantly rotating around the world at locations in Asia, Europe, and the Americas. In addition to advancing the work, the meetings also serve as a critical means for obtaining consensus among the participating parties, for establishing features and implementation schedules, for discovering and serving customers, and demonstrating substantive leadership. The processes are well-honed and have been highly successful globally in developing the world’s most significant network communications medium — the global mobile communications infrastructure.
January was an unusually slow 5G collaborative “breather” month — presaging a considerable array of meetings in February. It was marked by meetings of the 5G architecture group (SA2) in Incheon, the CODEC group (SA4) in Wroclaw, the mission-critical communications group (SA6) in Hyderabad, and the core E2E network & terminals group (CT1) virtually. So, who was present? Who contributed input documents to shape the results — arguably the most important metric of leadership?
The 5G Architecture group is one of its most important and active. It’s interim meeting in Incheon attracted 194 participants from 82 industry and government players, with the largest number from Huawei, Samsung, ETRI, Ericsson, LG, Nokia, and InterDigital. A number of other U.S. players were present, including both DHS’ FirstNet and the DOJ. There were 1831 document submitters from 72 industry players, led by Huawei/HiSilicon, Nokia/Nokia Shanghai-Bell, Ericsson, Qualcomm, Samsung, and Vivo. Among government agencies, DHS FirstNet had 3. The principal work item submissions included 38 key 5G architecture features, at the top of which were Network Automation for 5G, Proximity-based Services, Vertical and LAN Services, advanced V2X, Edge Computing, Non-Public Networks, Cellular IoT support and evolution for the 5G System, Access Traffic Steering, Switch and Splitting support, Policy and charging control, 5G multicast-broadcast services, multi-USIM devices, Specific services support, Location Services, Enablers for Network Automation for 5G, Enhancement of Network Slicing, and Enhancements to the Service-Based 5G System Architecture.
The 5G CODEC group is a highly specialized group that attracts those who offer multimedia platforms. Its Wroclaw meeting attracted 67 participants from 32 industry players, with the largest numbers from Ericsson, Qualcomm, Nokia, Huawei, InterDigital. Notably, Dolby, Apple, and even Facebook were present. There were 166 document submitters from 23 industry players, led by Qualcomm, Ericsson, Sony, Dolby, and Nokia.
The 5G mission-critical group is also highly specialized that deals with both National Security Emergency Preparedness (NSEP) capabilities as well as high resilience capabilities such as required for utility infrastructures. Its Hyderabad meeting attracted 68 participants from 33 industry and government players with the largest numbers from the Indian government, Samsung, ZTE, AT&T, Huawei, Motorola, and Qualcomm. There were 196 document submitters from 30 industry and government players, led by Samsung, Huawei/Hisilicon, Ericsson, ZTE, Convidia, and Nokia/Nokia Shanghai Bell. The Netherlands Police was the only government agency submitting contributions. The principal work item submissions included 13 key 5G mission-critical features, at the top of which were enabling Edge Applications, Mobile Communication System for Railways, application layer support for Factories of the Future, Mission Critical Services support over 5G System, architecture and information flows for Mission Critical Data, application layer support for V2X services, support for Unmanned Aerial System (UAS), Mission Critical services over 5G multicast-broadcast system, and location enhancements for mission-critical services.
The 5G E2E core network and terminals group met virtually among 35 participants from 23 industry and government players, including DHS FirstNet, with the largest numbers from Nokia, Qualcomm, Ericsson and MediaTek. There were 172 document submitters from 14 industry players with the largest numbers from Huawei/HiSilicon, Ericsson, Nokia/Nokia Shanghai Bell, MediaTek, Qualcomm and Blackberry. Almost all contributions were focussed on a single work item — the NAS protocol, which is used to manage the establishment of communication sessions and for maintaining continuous communications with the user equipment as it moves.
In the ever-important world of 5G security, some of the most significant work continues to be led by the UK government. The assertion out of Washington that somehow China controls or dominates the 5G security activity flies in the face of myriad security activities that proceed on a collaborative consensus basis, and at which the principal U.S. government security agencies and regulators have long been absent, much less active contributors.
Written by Anthony Rutkowski, Principal, Netmagic Associates LLC
Follow CircleID on Twitter
Read more here:: feeds.circleid.com/cid_sections/blogs?format=xml
By Hollis Kara
Can you believe it’s been a decade since we hosted our first ARIN on the Road event in Phoenix, AZ? We can’t either. And ever since that inaugural event we’ve been busy traveling all over our region to 58 locations in Canada, the Caribbean, and the United States.
ARIN on the Road events fit into the larger category of our outreach events. These events are designed to help potential, new, and existing customers understand all of the services that we provide to our community. We like to travel to locations where we don’t typically host our larger Public Policy and Members Meetings in order to bring our staff and leadership directly to all parts of our community throughout the region.
These events have provided our community a chance to get face-time with ARIN staff and leadership. These no-cost events offer the latest updates from ARIN to keep you informed about all the tools and services we provide to our customers, and how to use them. They typically last about half a day, and lunch is provided for all in attendance. Members of our staff present on various topics, such as IPv6, IPv4, RPKI, and more, with breaks for discussions.
“I have attended ARIN on the Road events as a community member, Advisory Council Chair, and now as a staff member of ARIN. The amount of information that is shared is incredible. These events allow different members of the ARIN community to come together for a day and share experiences to the benefit of all. The newly incorporated hands-on learning experiences will allow for even more opportunities to share and grow.”
– John Sweeting, Chief Customer Officer
A Slight Detour
Starting in 2020, we’ve decided to switch up the format of these events to better serve our community based on feedback from previous attendees. We’ll be providing more hands-on learning opportunities to increase participant engagement. We will guide you through the actual steps to make use of our account security features, resolve outstanding requests and other tickets, and answer questions that will allow you to make use of the full range of ARIN services.
Here’s a look at a sample agenda:
10:00 AM –
- Welcome and Getting Started
- ARIN’s Mission and Core Functions
- Making the Most of ARIN Technical Services
- Policy Development at ARIN
- IPv4 Today – Waiting List, Transfers, and more
- ARIN Security Services – DNSSEC, RPKI
- Community Engagement with ARIN
- ARIN Directory Services – RDAP, Whois, and WhoWas
- IPv6 Address Plans – What do you need and how do you get it?
- Q&A / Open Mic Session
- Ask ARIN (3:30 PM – 4:00 PM) – Optional opportunity for a one-on-one conversation with ARIN staff
We hope to see you at one of our upcoming ARIN on the Road events, with the first events kicking off in Memphis, TN and San Antonio, TX on the 25th and 27th of February. If you think your area could benefit from an ARIN Outreach event, let us know at firstname.lastname@example.org.
Read more here:: teamarin.net/feed/
By Joel Hruska
The enormous gap between what homeowners want from smart homes and what companies actually provide has come starkly into view. Spectrum, which formerly operated a home security service, has decided it doesn’t want to continue to do so. All current Spectrum security equipment will brick on February 5. Existing customers will be offered discounts on Ring and Abode systems, but will not
In theory, Spectrum’s security products should be compatible with other hardware, given that they use the common Zigbee protocol. Unfortunately, Spectrum has firmware-locked its own security products. They can’t “see” other Zigbee devices nearby, and other Zigbee devices can’t see them, either. Spectrum’s customers are understandably upset and have been calling on the company to release an open firmware before shutting down the service.
This isn’t the first time we’ve seen a high-end tech company pull a move like this. The problem, I think, is the fundamental mismatch in how customers think about these services versus how companies think about them. According to this site, the average urban homeowner moves every 9.7 to 12.5 years, depending on location. When you put in an appliance, you expect it to last a while. Humans being what they are, I’m certain somebody out there upgrades their washer every year, but most of us take an “If it ain’t broke, don’t fix it” approach.
Despite being run by ostensible humans, many tech companies haven’t figured out that longevity and consistency are features customers prize when they are installing products into their homes. The problem isn’t that Spectrum is shutting off its service — it’s that thousands of perfectly useful sensors and cameras are going to get dumped into landfills because Spectrum didn’t want to enable a feature.
This is a colossal waste, and it’s burning the very group of people a nascent market depends on in order to thrive: early adopters. At least one unhappy user spent $1,200 outfitting his home with sensors and equipment. Another reported that Charter / Spectrum had been pushing customers to pay for the entire service up-front when they signed up for service.
Legally, Spectrum is absolutely within its right to terminate services and throw the hardware away. This is a fine example of how something can be both outrageously stupid and completely legal. But this kind of capricious treatment runs the risk of souring people on the idea of smart homes in the first place. The hard truth is, there aren’t a lot of vendors you can trust. Some shutter, like Spectrum, while the successful ones break their word to the very adopters that helped them succeed in the first place.
If smart home companies continue to throw their own users away like garbage, they’ll quickly find themselves without a product to market. Homeowners expect these sorts of products to work and interoperate over a decade or more, not 1-3 years. Homeowners aren’t going to be willing to keep shucking out top dollar for incompatible systems. Hopefully, Spectrum will do the right thing and unlock its hardware before it goes dark.
But don’t hold your breath.
- PCMag’s Best Smart Home Security Systems for 2020
- Google Blocks Xiaomi Smart Home Devices from Assistant Following Camera Feed Mix Up
- If You Have a Smart TV or IoT Devices, Your Home is Leaking Data
- Home Builders are Reportedly Dumping Nest After Google’s Changes
Read more here:: www.extremetech.com/feed