Introduction
This article presents a visualization of a Twitter sample stream with a C#/.NET Core publisher application which captures the stream from Twitter, processes and re-publishes text and avatars through emitter messaging broker (also on GitHub), which allows broadcasting the stream to subscriber clients, written in Javascript.
![Image 1](/KB/cs/1159303/demo.gif)
Background
The architecture of the application is very simple, yet powerful. We have a publisher application which consumes tweets from Twitter API and publishes it in Emitter Broker. The broker, which handles multi- and broad-casting of the messages, is used to transfer the processed tweets from the publisher to the HTML5 consumers.
Tweet Publisher (C#/.NET Core)
- Establishes connections with Twitter Streaming API and Emitter Broker.
- Every time a new tweet is received, processes the tweet and publishes it into
tweet-stream
channel.
Javascript Client
- Subscribes to
tweet-stream
channel. - Every time it receives a tweet through
tweet-stream
channel, processes it and binds it to the view.
Tweet Publisher - C#/.NET Core App
Below is most of the code I had to write for the application that consumes Twitter Stream, processes it and republishes into Emitter.
- First, it uses a very nice, and .NET Core compatible Twitter API: https://github.com/linvi/tweetinvi to hook up the Stream and a
TweetReceived
event which will be invoke every time the message is received from Twitter. - It establishes the connection with Emitter and performs Twitter Authentication. If you want to try out for yourself, make sure to get the 4 security keys required to authenticate your app, which can be found here https://apps.twitter.com.
- We use a rate limiter, allowing us to have a steady and visually pleasing rate at which we show tweets to the user.
- For each tweet received, we skip if it’s a retweet and simply re-publish the
Text
and ProfileImageUrl400x400
in JSON format into tweet-stream
channel of Emitter.
public class Program
{
public static void Main(string[] args)
{
var emitter = Connection.Establish();
Auth.SetUserCredentials(
Environment.GetEnvironmentVariable("CONSUMER_KEY"),
Environment.GetEnvironmentVariable("CONSUMER_SECRET"),
Environment.GetEnvironmentVariable("ACCESS_TOKEN"),
Environment.GetEnvironmentVariable("ACCESS_TOKEN_SECRET")
);
var limiter = new Throttle(1, TimeSpan.FromMilliseconds(350));
var stream = Stream.CreateSampleStream();
stream.AddTweetLanguageFilter(LanguageFilter.English);
stream.FilterLevel = Tweetinvi.Streaming.Parameters.StreamFilterLevel.Low;
stream.TweetReceived += (sender, t) =>
{
if (t.Tweet.IsRetweet)
return;
var ct = new CancellationToken();
limiter.Enqueue(() =>
{
emitter.Publish(
"IsQ7z18uGEFpjOJpt4K6ij49klT3PGzu",
"tweet-stream",
JsonConvert.SerializeObject(new
{
avatar = t.Tweet.CreatedBy.ProfileImageUrl400x400,
text = t.Tweet.Text
}));
}, ct);
};
stream.StartStream();
}
}
Tweet Publisher - Deploying with Docker
We are going to deploy our little .NET Core publisher using Docker. Docker containers wrap a piece of software in a complete filesystem that contains everything needed to run: code, runtime, system tools, system libraries – anything that can be installed on a server. This guarantees that the software will always run the same, regardless of its environment.
![Image 2](data:image/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==)
One of the easiest ways to get started is to use a Dockerfile along with Docker Hub Build Server which would build an image we could deploy on the target machine. Docker Hub allows us to automatically build a Docker Image (think of a Virtual Machine Image here) which we could deploy. The image can be triggered to automatically re-build on every commit to the git repository.
Our docker file, as shown below, does several things and essentially represents a script which is used to create the Docker Image:
- It starts with the Docker image provided for .NET Core by Microsoft.
- Updates the apt-get sources and clones the Git repository.
- Performs
dotnet restore
which essentially does a nuget restore
and loads all the dependencies. - Performs
dotnet publish
which creates final binaries for our image. - Hooks a
deploy.sh
script, which will be executed every the image starts. This script simply runs our application.
FROM microsoft/dotnet:1.0.0-preview2-sdk
MAINTAINER Roman Atachiants "roman@misakai.com"
# Make sure we have S3 & additional libraries
RUN apt-get update -qq \
&& apt-get install -y git \
&& mkdir /tmp/emitter \
&& cd /tmp/emitter \
&& git clone "https://github.com/kelindar/twitter-stream.git" "/tmp/emitter" \
&& cd /tmp/emitter \
&& dotnet restore -v Minimal \
&& cd /tmp/emitter/src/Server \
&& dotnet publish -c Release --output /app \
&& rm -rf /tmp/emitter
# Application will be in app folder
WORKDIR /app
ADD deploy.sh /
CMD ["/bin/bash", "/deploy.sh"]
Once the image is built, we can start it by typing docker run -it kelindar/twitter-stream
, assuming you have docker installed. It will automatically download the image from Docker Hub and run it for us interactively.
Javascript Client
Similarly to the C# app we just wrote, the client is extremely simple and straightforward. In fact, all of the javascript (excluding dependencies) is shown below. The client does several things:
- Connects to emitter publish/subscribe service.
- Sets up our ViewModel using VueJS.
- Once we’re connected to emitter, it subscribes to
tweet-stream
channel. - Every time a message is received, we push it into the
data
property of our ViewModel, which is bound to the HTML DOM.
var emitter = emitter.connect({
secure: true
});
var key = 'jtdQO-hb5jfujowvIKvSF41NeQOE8IoF';
var vue = new Vue({
el: '#app',
data: {
messages: []
}
});
emitter.on('connect', function(){
console.log('emitter: connected');
emitter.subscribe({
key: key,
channel: "tweet-stream"
});
})
emitter.on('message', function(msg){
if (vue.$data.messages.length >= 8){
vue.$data.messages.shift();
}
vue.$data.messages.push(msg.asObject());
});
Most of the UI here comes from one of my previous articles where emitter was used to create a simple group chat. Feel free to check the live demo and the corresponding CodeProject Article if you’re interested.
History
- 12/04/2016 - Initial Version of the article