What YOU Need to Know About HLS: Pros and Cons

January 15th, 2016 by Chris Allen

R5P_HLS blog post

HLS, or HTTP Live Streaming is a video streaming format first introduced by Apple in May, 2009. It’s a format that breaks streams into small file-based segments made available for download over HTTP. It now is a widely supported format for viewing streams in almost real time. I say almost only because the protocol, by its very nature, introduces a lot of latency.

We’ve been getting a lot of questions and inbound interest from our customers about HLS. What are the advantages of using it? What are the disadvantages? How does HLS compare to WebRTC? Will Apple approve apps that don’t use HLS for streaming? This post is an attempt to address some of these questions.

Advantages

There are many reasons you would want to use HLS for your live streams, and this is why we recently added the support to Red5 Pro.

Ubiquity

First of all, HLS is widely supported. Although originally conceived by Apple for Quicktime, iOS and the Safari browser, it’s now implemented on virtually every browser out there. As you can see, the leading browsers support it. Now of course most of them support a comparable standard called MPEG DASH as well, but since Apple Safari and iOS devices don’t support it, we think HLS is currently a better choice.

Adaptive Bitrates

Another huge advantage of using HLS is that it allows for clients to choose from a variety of quality streams depending on the available bandwidth.

 

Disadvantages

So if it does all that, why wouldn’t I want to use HLS for my live streaming app?

Terrible Latency

It turns out that while HTTP Live Streaming was designed to deal efficiently with multiple quality streams, it wasn’t built for delivering video quickly. Practically speaking, HLS introduces at least 20 seconds of latency (often more!) in your streams.

Here’s why; HLS requires three segments in queue before it will allow playback, and the segments are divided by the keyframes in the video. The only way to create a super low latency stream (let’s say under one second) with HLS is to encode the video going out with a keyframe happening every 250 ms. The HLS playlist moving-window would be four items long with a segment duration of quarter of a second.  This would of course create high bandwidth video, add to the number of HTTP calls happening (at least four per second), and put additional load on the server.

The whole point of keyframes in video protocols like h.264 is to reduce the number of times you need to send a full frame of image data. With the above scenario, you might as well be sending the frames as a series of JPEG images in sequence. There’s a lot more to this, like the fact that media has to be packaged in 188 byte packets which creates added overhead when you do it too much, but hopefully now you’ve got the gist of it: HLS is a poor choice when it comes to low latency video streaming.

No Publishing

HLS is a subscriber-only protocol. Unlike WebRTC, which has a spec for publishing from a browser, HTTP Live Streaming only supports playing streams. If you want to publish a live video stream for a device, you simply have to look for other technology to do this. Luckily with Red5 Pro, we have alternative SDKs for mobile that allow you to create publishing apps that utilize RTP; you can then relay those streams over HLS for folks to view these streams right in their browsers. You can check out our HLS player example using video.js on GitHub. We are also in development on full WebRTC support with Red5 Pro that will include a JavaScript SDK. This implementation will feature out-of-the-box tools like a WebRTC Publisher and a player that supports WebRTC, HLS and RTMP (as a fallback), so stay tuned for that update as well.

Apple iOS HLS App Rules

Another question that comes up when our customers get ready to submit their iOS apps to the App Store is: will Apple reject my app if it’s not using HLS? As many of you know, our SDK uses RTP streaming for iOS, and Apple has some strange requirements that all apps must use HLS for streaming. That’s not quite true however. Apple states the following in their App Store Submission Guidelines:

  • “9.3 Audio streaming content over a cellular network may not use more than 5MB over 5 minutes.”

  • “9.4 Video streaming content over a cellular network longer than 10 minutes must use HTTP Live Streaming and include a baseline 192 kbps or lower HTTP Live stream.”

What we’ve also found is that if the app is a communication app–meaning that you have some form of two-way communication like Periscope has with live chats–then they tend to group the app in a different category. Apple also considers video calling apps like Skype to be in a different category, and the live streaming restrictions of having to use HLS don’t apply. The other good news is that as the popularity of apps like Periscope and Meerkat continues, Apple is getting used to the idea of live real-time streaming apps, and is gradually becoming more and more flexible with the restrictions.

So with that in mind, because of HLS’s high latency, Apple will approve apps that use other protocols if there’s a need for real-time communication. We simply recommend making a note of why you can’t use HLS when you submit your app.

Summary

As you can see, the ubiquitous support of HLS on a variety of browsers, mobile phones, and operating systems makes it a great choice for distributing your streams to the most amount of viewers. However, since it’s a rather slow protocol, if you are building any kind of app that relies on near real-time communication, you should look at other options. Finally, while Apple’s rules do seem quite rigid when it comes to their iOS streaming requirements, they are actually flexible when the need for something else is justified. What are you thoughts; are you currently using HLS in your apps? Have you submitted a non-HLS based streaming app to Apple’s app store? How did that go? Let us know in the comments.

 

, ,

He’s BAAAaaaack!

November 10th, 2015 by Paul Gregoire

Screen Shot 2015-11-10 at 4.11.15 PM

Why on earth would anyone choose to rejoin a company where they previously worked? In retrospect, the answer seems quite simple, though of course, taking the plunge is always a daunting task. I assume the statistics do not favor a return to a previous workplace after an extended hiatus, but I could certainly provide a plethora of reasons why I returned to Infrared5 when presented with the opportunity to do so. For me, and in any and all endeavors, the work has to be both intellectually stimulating and challenging. Software development can often be repetitive and monotonous, but fortunately that never was (and still isn’t) the case at Infrared5. Of near equal importance are the workmates that surround me; without a high quality, collaborative and generally pleasant team, even the most stimulating work can lose its shine.

So to that end, here is a short checklist I used when making the decision:

1. Do I enjoy my interactions with my teammates? Check!

2. Do I have respect for them and their abilities? Check!

3. Lastly, Infrared5 wanted to pay me to write code all day and work on Red5! Well that’s a no brainer–count me in!

While working for another company, I had joined the Red5 Pro beta just after it was announced on Twitter. A “Pro” version or Red5 was something Dominick and I had chatted about on a few occasions back in the day–there may or may not have been beer involved. Despite the casual conversations, we both thought there could be immense long-term value in providing the Red5 community with what they were always asking for and talking about on the users list and social media. Providing the ability to stream from any device to any device has always interested me. Alas, I had the desire to turn intrigue into application by helping to create a software which could do just this. Red5 Pro offers Flash streaming, HLS streaming, and RTSP streaming; these technologies are still in-demand and are not interoperable on existing platforms, except on Red5 Pro.

In the near future, the public can expect some major disruption on the streaming media front from Red5 Pro. It should come as no surprise to the community that we’re adding the new hotness of technology, WebRTC, to our existing stack. If this addition isn’t “For the Win,” I don’t know what is. I have no qualms about saying that the suite of tools which comprise Red5 Pro will help shape the future of the streaming market–and I couldn’t be happier with my decision to return.

-Paul

 

, , , ,

First Annual Intel Innovators Summit

November 5th, 2015 by Chris Allen

Intel Innovator Summit 2.JPG

 

I just got back from the Intel Innovators Summit in Santa Clara, CA where I serve as a current Intel Software Innovator. Intel brought together some of the top programmers, makers and creative technologists from around the world to show us some of the innovative things they are working on. While I can’t share many of the details of what was presented, to me, the most valuable aspect of the summit was having the fantastic opportunity to connect with so many talented developers in one space.

During the event, Intel split up the topics between RealSense and IoT. RealSense, for those who don’t know, is a platform built around 3D depth cameras. As mentioned above, due to an NDA I can’t share everything Intel is up to, but much of what they are doing has been released publicly, and there are public SDKs that anyone can download to start thinking with the devices. As a quick aside, here’s a link to a video that Infrared5 put together for a contest on perceptual computing (which is actually the basis of RealSense) as a catalyst for a truly immersive game play experience at the 2013 Intel Ultimate Coder Challenge. While the front-facing cameras on Windows PCs offer quite a few interesting options for Minority Report style input and games, I find the rear/world-facing r200 cameras on Android devices much more intriguing. The possibilities surrounding the combination of the devices with IoTs are enormous; Imagine unlocking doors just by a camera recognizing your face, or creating wearables that interact with the world with 3D data.

In retrospect, IoT was really the topic that got me excited at this event. Though I don’t consider myself a Maker, Infrared5 has engaged in a significant number of projects with internet connected sensors. Our customer SICK, for example, utilizes Red5 for live streaming of data coming from their sensors. I’m really excited to get my hands on an Intel Edison board after seeing firsthand how the device can seamlessly enable live streaming across a variety of products.

SparkFun_Edison_Boards-14

The fact that it’s so small–and relatively powerful– introduces all kinds of exciting use cases beyond mobile phones. Rest assured, we will be experimenting and sharing with you guys what we and the other Innovators do with Red5 Pro and these devices.

One of my favorite activities of the summit was the little contest Intel ran called the Spark Tank, which was undoubtedly inspired by the hit TV show Shark Tank. Intel kicked off the exercise by splitting the Innovators up into groups with the task of inventing a project which we then pitched to the judges. The judges consisted of the Intel Innovator program as well as executives from the company. I must say, all the teams came up with compelling experiences–everything from doors that scanned your face in order to allow entry to stuffed animals with medical sensors embedded within.

Our project consisted of a military application to improve communication within squads and leveraged the Red5 Pro platform with the Edison board. Each team had to do a skit or a spontaneous commercial for their product and present on the benefits of their concept. For ours, we ended up cutting out cardboard rifles and paper goggles and then taped together gesture/motion control MYO armbands that let you wirelessly control connected devices. It was rather dramatic, as we had our team get shot and die on stage. While we won the “Most Likely to Get the Blind into the Military” award, we clearly should have won funniest skit for the most serious topic.

Intel Innovator Summit 1.JPG

It’s fantastic to be a part of such a talented group of developers, and I’m honored to be able to represent Infrared5 in the Intel Innovators group. Kudos to Intel for having one of the best developer outreach programs out there! I’m excited about all the possible collaborations that will emerge from the Intel Innovator’s Summit, and of course, we can’t wait to see what these guys do with live streaming and Red5 Pro.

, , , ,

What We’ve Quietly Been Working On: Red5 Pro – Going Back to Our Roots

November 4th, 2015 by Chris Allen

red5pro_live_broadcast.png

 

The Times They Are A Changin’

As you might have noticed, we recently updated our website to better reflect our new direction at Infrared5. We are now focused on the Red5 Pro Server and SDKs for iOS and Android that enable developers to build experiences like Meerkat or FaceTime in a matter of minutes. Yes, you read that correctly – the ability to create mobile streaming applications in minutes. We originally started this company because of the tremendous reception and interest in the Red5 Open Source Media Server. For those that aren’t familiar with the project, our initial team, composed of John Grden, Paul Gregoire, Dominick Accattato and myself, worked with other developers around the world to reverse engineer the RTMP protocol and create an open source alternative to Macromedia’s Flash Communication Server. This project eventually became Red5. Two years later as the project grew, we noticed strong demand from developers who needed custom work and consulting on Red5 – so much so that we decided to quit our day jobs and start Infrared5. Over the years though, our focus drifted away from exclusively building live streaming solutions with Red5.

Games and Brass Monkey

Our developers at Infrared5 have always been interested in disrupting the present and pushing the boundaries with new technologies. One of these instances was the Unity game engine. Andy Zupko and John Grden really pushed us in this direction as early adopters of the platform. We built many great games on Unity including the Star Wars Trench Run, Hasbro’s Game of Life Zapped Edition, and most recently the augmented reality Force Trainer feature in the official Star Wars app. Even though we are no longer focused on games, our passion for game design and the unique experiences they enable really influence our product design. We want to make using Red5 Pro fun and enjoyable for developers, which in many respects isn’t far off from the goal of a good game.

b-pre-connect

During this time Rebecca led Infrared5 not just in games projects, but also on IoT and streaming projects that leveraged the open source Red5. Another project that Infrared5 invented and spun off was our smartphone-as-a-game (SAAG?) controller product, Brass Monkey. I moved over to lead that company as CEO in 2010, but eventually I came back to Infrared5 full time, as we weren’t able to effectively convince people to pay for smartphone controlled, browser-based games. Note though, we kept the technology, and it’s now part of Red5 Pro as the Second Screen SDK.

Going Back To Our Roots

After Steve Jobs announced the demise of Flash with the lack of support in iOS, and the eventual decay of support for Android, we came to the conclusion that we would solve the “Get off of Flash Problem” for live streaming mobile apps. We heard from a lot of our consulting clients that they wanted this, and instead of trying to build custom solutions over and over again, we decided Red5 needed an upgrade. Mobile SDKs for RTMP are mostly fragmented, hard to use, clunky and generally just a mess. We have now made it our mission to make building a live streaming app for iOS and Android efficient and intuitive. Whether it’s a one-to-many live broadcasting app like Periscope, a many-to-many conferencing app, or a one-to-one video chat application, we want to make it so incredibly simple that any developer can do it.

The Future: WebRTC, Second Screen, IoT

While migrating existing live streaming Flash apps to our new platform is super helpful, what we are most excited about is our vision for the myriad applications of the Red5 Pro technology.

An ever-increasing number of browsers are adding support for WebRTC; heck, even Microsoft Edge is getting there! We think this is clearly the future for in-browser streaming, and we are currently working on making Red5 Pro speak this protocol. We see the Red Pro Server as the underlying hub that is able to talk to all different streaming mobile apps and browser apps with minimal latency and outstanding performance.

However, the phones we carry in our pockets, the laptops sitting on our desks, and the tablets we browse while sitting on the couch are truly just the beginning. Other devices with cameras that can connect to the internet are the next big thing in live streaming. We are thrilled to make Red5 Pro integrate with all of these Internet of Things devices. The possibilities are endless: imagine fully immersing yourself in a live concert via your VR headset streamed live from a 3D camera at the venue, or enabling live video streaming among military troops over mesh networks going to their AR headsets. There are countless things for developers to build in this space, and we’re excited to see how we can power them through Red5 Pro.

Finally, we think that the second screen experiences like what we started with Brass Monkey have huge potential for changing the way people interact with technology. Not only can you as a developer turn phones into game controllers, but you can also create new banking software that enables your phone to interact with and take away information from a screen in bank branches of the future.

red5pro_secondscreen

…And There’s More!

Of course we can only think of so many scenarios of how our tech can be applied. The true innovation will be done by what you as a developer create with it. What would you build with Red5 Pro? Let us know in the comments. Much more coming your way soon!

, , , ,

The benefits to working in an Agile process

October 7th, 2013 by Jes Palmer


Here at Infrared5, our team of designers and developers have dedicated a lot of effort to fine-tuning a process that works most effectively and efficiently for both the client and our team. We want to share why we feel working in an Agile process has not only been successful on our projects in the past, but also want our future clients to have an understanding of how this works in their benefit on upcoming projects. Read the rest of this entry »

,

IR-top-5 Design and Development Tools

September 13th, 2013 by Adam Doucette

Here at Infrared5, we like to share our knowledge and insights on what is the latest and greatest in the world around us. We are starting a new monthly feature on the Infrared5 blog, our IR-top-5 (yeah I know…so clever). We will poll our team on certain topics and give them free rain to share their individual 2 cents on the topic. It could range from top-5 MMO games to top-5 IPA’s. To kick things off, I asked our team of Developers and Designers to share their picks for tools they are currently using which help work better. Besides pretty much getting an auto-reply from everyone with the word “BEER” in it (yes, in caps), here are what the Infrared5 team had to share about the top-5 design and development tools.

Read the rest of this entry »

, , , , ,

My Thoughts on Glass

August 27th, 2013 by Chris Allen

I recently took a trip down to NYC a few weeks ago to pick up my new Google Glass. I figured now that we’ve had a bit of time to play with it here at Infrared5 and Brass Monkey, it’s time to write a post on our findings and what we think.

Read the rest of this entry »

, , , , ,

The Making of MassROUTE, Part I: Inception

August 5th, 2013 by Todd Anderson

Arduino + Bi-Color LEDs

Prior to moving to our beautiful new office here at Infrared5, we were located just off Centre Street in Jamaica Plain. The location was not only convenient for delicious food choices, but also for being located along the MBTA 39 Bus Route.

I prefer to ride my bike into work, but if for some reason that is not possible, I take public transportation. One reason that would prevent me from riding my bike is weather. If you are familiar with winters here in Boston it’s not particularly fun to stand outside in freezing wind and snow waiting for a bus that – let’s face it – doesn’t run on a schedule. In fact, we have a little pet name for the buses that run around here: we call them ‘Banana Buses’ because they come in bunches, leap frogging down Centre in sets of 3.

MBTA scheduling concerns aside, we are fortunate enough to be provided with real-time data from Massachusetts Department of Transportation which, I must say, is quite accurate.

We had been playing around with the real-time bus data for a bit, making various apps and even just using the service as a jumping off point for learning more about a language or library through data integration. Due to a recent fascination with microcontrollers and circuitry, I thought it was about time we have some physical indicator within the office that would notify us of the next approaching bus so we didn’t have to keep checking one of the several apps we built. As such, a project we call MassROUTE was born.

Read the rest of this entry »

, ,

Kiwi Katapult Revenge Case Study by Intel

July 29th, 2013 by Adam Doucette

Earlier this year, Intel invited Infrared5 to compete in its Ultimate Coder Challenge: Going Perceptual, a groundbreaking contest that provided participants with an Ultrabook device, Creative Interactive Gesture Camera development kits, a still-evolving perceptual computing SDK, and all of the support possible for letting their imaginations run rampant. With its focus on sensor-based input, using Brass Monkey technology seemed a natural complement to perceptual computing, but the question surfaced of how to mesh the two. William Van Winkle recounts the story our team experienced in this challenging but fulfilling opportunity. As always, we welcome all questions and feedback. Enjoy!

http://software.intel.com/en-us/articles/infrared5-case-study

, , , , , ,

What our team at Infrared5 is watching in emerging technology?

July 10th, 2013 by Adam Doucette

Here at Infrared5, we know we have one of the best development teams in the business. Our team of designers and developers know the ins and outs of (and have even written textbooks on) programs like Flash, Red5, Unity, and many other programs we use on a daily basis. But being in an industry where technology changes often and quickly, our team realizes they have chosen careers that require lifelong learning and understand the need for adapting to new technologies constantly. This past week I decided to ask some of our team members what technology they are keeping their eyes on and are excited to learn more about.

Read the rest of this entry »

, , , , , , , , ,

« Previous Entries