Posted by Introduction and Resources
Welcome to my 7-post series on Complex Event Processing with Mircosoft StreamInsight. Enjoy!
Microsoft StreamInsight is a very lightweight, powerful and for .NET developers easy to use Complex Event Processing (CEP) engine. It uses .NET LINQ queries to write queries on data streams.
With minimal effort, StreamInsight allows you to answer questions such as “what is the one minute aggregate of my value X grouped by dimension D”. This series of blog posts explains the concepts behind CEP and StreamInsight and implements a sample application that can be used to analyze live data from Twitter.
Index of this blog series:
Part 1 (03.10.13): CEP with StreamInsight and Twitter 1:
Introduction and Resources
Part 2 (06.10.13): CEP with StreamInsight and Twitter 2:
What are CEP and StreamInsight
Part 3 (13.10.13): CEP with StreamInsight and Twitter 3:
StreamInsight Programming
Part 4 (20.10.13): CEP with StreamInsight and Twitter 4:
Reactive Programming
Part 5 (27.10.13): CEP with StreamInsight and Twitter 5:
The Sample Application 1
Part 6 (03.11.13): CEP with StreamInsight and Twitter 6:
The Sample Application 2
Part 7 (10.11.13): CEP with StreamInsight and Twitter 7:
The Sample Application 3
The best resources that I found:
Video: “What’s new in Complex Event Processing using StreamInsight for SqlServer 2.1” by Torsten Grabs, Principal Program Manager Lead, Microsoft
http://www.youtube.com/watch?v=2k1AO5Wr_kk
Video: “My Campus Walk: I walk, StreamInsight analyzes, Worldwide Telescope visualizes.” Interesting project about visualization of a walk path from geo-spatial data processed by StreamInsight.
http://www.youtube.com/watch?v=Zi7JJmC4Nxw
Video: “Heat mapping H5N1 Outbreaks with StreamInsight and Binq Maps”. Really awesome demo application that feeds processed stream data to a Bing Maps application.
http://www.youtube.com/watch?v=cctmJZZDBlg
Tools
A tool I absolutely recommend, especially for StreamInsight Development, is LINQPad (http://www.linqpad.net/). LINQPad includes the following StreamInsight sample packages that can be executed and played with:
- 101 StreamInsight Samples
- Hitchhikers Guide to StreamInsight
- Performance Counter Samples
- Reactive Extensions
- LINQ in Action
Important Note:
If you use the free version of LINQPad you will not get any intellisense or auto-complete functionality. In my opinion this makes it useless. As soon as I bought a personal license (about USD30), LINQPad turned into one of my favorite dev tools of all times. It is extremely lightweight and understands C#, VB, F#, SQL, LINQ, ADO.NET, StreamInsight, etc.
Samples
TwitterStream Sample: The application that we are going to develop in the following blog posts:
Hitchhikers Guide to StreamInsight Queries: Very comprehensive guide to writing queries in StreamInsight. Code is included in the codeplex download below. Samples are included in LINQPad.
http://blogs.msdn.com/b/streaminsight/archive/2012/08/01/a-hitchhiker-s-guide-to-streaminsight-2-1-queries.aspx
StreamInsight Product Team Samples 2.1
http://streaminsight.codeplex.com/releases/view/90143
StreamInsight Product Team Samples 2.0
Very important collection of samples such as: TextFileReaderAdapter, TextFileWriterAdapter, SqlAdapter, WcfAdapter and lots of applications.
http://streaminsight.codeplex.com/releases/view/46435
JAhlen StreamInsight Example Adapters v2.1
Stock trading sample application.
http://streaminsight.codeplex.com/releases/view/97583
Blogs/MSDN/Technet/Forums:
Blog: Official Blog of the Microsoft StreamInsight Dev Team
http://blogs.msdn.com/b/streaminsight/
Blog: StreamInsight explained to non-programmers
http://social.technet.microsoft.com/wiki/contents/articles/14437.streaminsight-for-non-programmers.aspx
Blog: Getting Started with StreamInsight 2.1
http://blogs.msdn.com/b/streaminsight/archive/2012/09/18/getting-started-with-streaminsight-2-1.aspx
Technet: Microsoft StreamInsight
http://technet.microsoft.com/en-us/library/hh750618(v=sql.10).aspx
MSDN: Microsoft StreamInsight
http://msdn.microsoft.com/en-us/library/hh750618(v=sql.10).aspx
MSDN Forum: SQL Server –> Microsoft StreamInsight
http://social.msdn.microsoft.com/Forums/sqlserver/en-US/home?forum=streaminsight
Next Post: