How to Implement Real Time Data for a Web Page

How do they make real time data live on a web page?

I did it with JavaScript timer set execution in milliseconds, each time timer executed function that queried Server with Ajax and returned value(possibly JSON format), then you you update your field with the value. I did it each 5 sec and it works perfectly. In ASP.NET I think it called Ajax Timer Control.

Display real time data on website that scales?

Dont use serverless functions(Lambda/Azure functions)

Although I am a huge fan of serverless functions, and currently running a full web app in Lambda, I don't think its needed for your use case and doesn't make sense economically. As you've answered in the comments, each airline will not write directly to the database, they'll push to an API, meaning you are explicitly told when flights have changed. When an airline has sent you new data you can simply propagate this to all the browser endpoints via websockets. This keeps the design very simple. There is no need to artificially create a database event that then triggers a function that will then tell you a flight has been updated. Thats like removing your doorbell and replacing it with a motion detector that triggers a doorbell :)

Cost

Money always deserves its own section. Lambda is more of an economic break through than a technological one. You have to know when its cost effective. You pay per request so if your dealing with a process that handles 10,000 operations a month, or something that only fires 1,000 times a day, than lambda is dirt cheap and practically free. You also pay for the length of time the function is executing and the memory consumed while executing. Generally, it makes sense to use lambda functions where a dedicated server would be sitting idle for most of the time. So instead of a whole EC2 instance, AWS provides you with a container on demand. There are points at which high requests rates and constantly running processes makes lambda more expensive than EC2. This article discusses how generally its cheaper to use lambda up to a point -> https://www.trek10.com/blog/lambda-cost/ The same applies to Azure functions and googles equivalent. They are all just containers offered on demand.

If you're dealing with flight information I would imagine you will have thousands of flights being updated every minute so your lambda functions will be firing constantly as if you were running an EC2 instance. You will end up paying a lot more than EC2. When you have a service that needs to stay up 24/7 and run 24/7 with high activity that is most certainly a valid use case for a dedicated server or servers.

Proposed Solution

These are the components I would use below:

Message Queue of some sort (RabbitMQ or AWS SQS with SNS perhaps)
Web Socket Backend (The choice will depend on programming language)
Airline input API (REST,GraphQL, or maybe AWS Kinesis Data Firehose)

Sample Image

The airlines publish their data to a back-end api. The updates are stored on a message queue and the web applicaton that actually displays the results to users, via websockets, reads from the queue.

Scalability

For scalability you can run the websocket application on multiple EC2 instances (all reading from the same queuing service) in an autoscaling group, so with extra load more instances will be created automatically hence the name "autoscaling". And those instances can sit behind an elastic load balancer. Lots of AWS documentation on how to do this and its their flagship design pattern. If you use AWS SQS you don't have to manage the scalability details yourself, aws handles that. The only real components to scale are your websocket application and the flight data input endpoint. You can run the flight api in an autoscaling group as well but AWS does offer an additional tool for high traffic data processing. I detail that below.

Testing Scalability

It would be fairly easy to have a mock airline blast your service with thousands and thousands of fake updates and on the other end you can easily run multiple threads of selenium tests simulating browser clicks and validating that the UI is still operational.

Additional tools

If it ends up being large amounts of data, rather than using a conventional REST api for your flight update service you could consider a service AWS offers specifically for dealing with large amounts of real time updates (Kinessis Data Firehose) https://aws.amazon.com/kinesis/data-firehose/ But I've never used it.

How to implement real time data for a web page

SignalR

This is the answer I'm most excited to share, because it represents a much cleaner implementation that is lightweight and works well in today's mobile (data constricted) environment.

There have been several methods over the years to provide "realtime" pushing of data from the server to the client (or the appearance of pushing data). Rapid Short Polling (similar to my AJAX based answers), Long Polling, Forever Frame, Server Sent Events, and WebSockets are different transport mechanisms used to achieve this. SignalR is an abstraction layer capable of selecting an appropriate transport mechanism based on the client and server's capabilities. The best part of using SignalR is that it's simple. You don't have to worry about the transport mechanism, and the programming model is easy to understand.

I'm going to define a SignalR hub, but just leave it empty.

public class GameHub : Hub
    {
    }

When I add data to the "database", I'm going to run the below bit of code. If you read the question, you'll see I commented it out in the "create" form. You'll want to uncomment that.

var context = GlobalHost.ConnectionManager.GetHubContext<GameHub>();
context.Clients.All.addGame(game);

Here's my page code:

<h1>SignalR</h1>
        <asp:Repeater runat="server" ID="BoardGameRepeater" ItemType="RealTimeDemo.Models.BoardGame">
            <HeaderTemplate>
                <table border="1">
                    <thead>
                        <tr>
                            <th>Id</th>
                            <th>Name</th>
                            <th>Description</th>
                            <th>Quantity</th>
                            <th>Price</th>
                        </tr>
                    </thead>
                    <tbody id="BoardGameTblBody">
            </HeaderTemplate>
            <ItemTemplate>
                <tr>
                    <td><%#: Item.Id %></td>
                    <td><%#: Item.Name %></td>
                    <td><%#: Item.Description %></td>
                    <td><%#: Item.Quantity %></td>
                    <td><%#: Item.Price %></td>
                </tr>
            </ItemTemplate>
            <FooterTemplate></tbody></table></FooterTemplate>
        </asp:Repeater>
        <script src="//ajax.googleapis.com/ajax/libs/jquery/2.1.1/jquery.min.js"></script>
        <script src="Scripts/jQuery-1.6.4.min.js"></script>
        <script src="Scripts/jquery.signalR-2.1.1.min.js"></script>
        <script src="signalr/hubs"></script>
        <script type="text/javascript">
            var hub = $.connection.gameHub;
            hub.client.addGame = function (game) {
                $("#BoardGameTblBody").append("<tr><td>" + game.Id + "</td><td>" + game.Name + "</td><td>" + game.Description + "</td><td>" + game.Quantity + "</td><td>" + game.Price + "</td></tr>");
            };
            $.connection.hub.start();
        </script>

And the code behind:

protected void Page_Load(object sender, EventArgs e)
        {
        BoardGameRepeater.DataSource = Application["BoardGameDatabase"];
        BoardGameRepeater.DataBind();
        }

Notice what's happening here. When the server calls context.Clients.All.addGame(game); it's executing the function that's been assigned to hub.client.addGame for every client that is connected to the GameHub. SignalR is taking care of wiring up the events for me, and automatically converting my game object on the server to the game object on the client. And best of all, there's no network traffic back and forth every few seconds, so it's incredibly lightweight.

Advantages:

Very light on the network traffic
Easy to develop, but still flexible
Doesn't send viewstate with the request
Doesn't poll the server continuously.

Note, you could add a function on the client for editedGame for pushing changed data out to the client easily (same for delete).

Displaying data in real-time on web page

The term is Comet, and it's basically just a block of code setup to run forever and poll (send an HTTP request) at intervals to get new data back, if any, populating an existing area with the new data.

You can read more about it:

http://en.wikipedia.org/wiki/Comet_%28programming%29

A tutorial ( not the best style of code ):

http://www.webreference.com/programming/javascript/rg28/

Stackoverflow question with an example:

How do I implement basic "Long Polling"?

How does one do realtime updates of a web page?

You can use Comet.