The Internet Of Things IoT and Machine Learning AI (Video Lesson)

I’ve actually got a couple of devices plugged into me. Right now I’ve got this band. This is real. It’s monitoring my heart rate. But I’m also a Type I diabetic. And I’ve been diabetic for 20 years. One of the things that every diabetic does when they become diabetic is they try to solve this problem usually with Microsoft Excel; but at some point they try to figure out, well, I’m a diabetic and I’m an engineer, let’s plug some wires together and see what we can do. So when we say the Internet of Things, I think it’s great that you, the normal sugared people, you know, you have your FitBits, and we think that’s adorable, but I have an insulin pump and that insulin pump is plugged in. There’s the insulin pump. And it’s plugged into my arm. It’s 24 hours a day. And that talks to a system. Here is a CGM. That is my actual live blood sugar right now. You see how it’s kicking up there a bit? That’s because of stress.

That’s actually plugged in down here in an implant in my side. Sorry to show you my chubbiness. And this is real. When you hear Internet of Things, you think Fitbit and you think Raspberry Pi, but I think this. And we’ve plugged this into some cloud systems that I want to share with you. While the Internet of Things is fun, things like Raspberry Pi, this is great, this is a Raspberry Pi, this is an Internet of Things enabled one. We’ve actually got C code on this talking to this temperature sensor. So what does that temperature sensor look like? Let me switch over to some code here and show you.

This is some C code inside of Visual Studio. This is the low-level stuff. C code makes your eyes blur a little bit sometimes. What you need to know, people who are wearing suits is you’ve got IoT Hub and you have Azure devices. So the takeaway there is that that’s cloud-enabled. But for the developers that think this is crazy, I want you to notice that we’ve actually got a Visual Studio plugin to do remote debugging of things like Raspberry Pis in Visual Studio. So whether it be high level, updating hundreds of thousands of objects, or whether it’s low level and doing remote GDB debugging to a Raspberry Pi, we can do that. Now let’s talk about my system. Let me bring back to the slides here for a moment. This is overwhelming. This is Azure, and all these icons, all these people doing all these things. Let’s talk about me and the system that I’m building. That’s me in my shirt. I’ve got my bands. The band goes to Microsoft Health. That goes ultimately into storage.

My glucose system actually goes up into an API, and I pull that data in as well because that’s personal to me. I’ve got my heart rate over years. I’ve got my blood sugar for years. What can I do with that? I’m going to show you some visualizations, some graphs and I’m going to plug some things into Office and see if Azure machine learning can answer some really interesting questions about me and my health.

And this is all live and this is all real. So let’s come back over to this machine here. This is a system called Nightscout. This is an open source node application that is running in Azure, and this talks to my CGM, my Continuous Glucose Meter. And I want you to point out a couple of things here. This is the first presentation today. And this is now. That is a real-time system that is showing my blood sugar.

And one of the things that happen is not just that I drink orange juice, which, by the way, I have in case I pass out, but also stress. Stress dumps glucose into the system. And this is just one of the things that I have to deal with when I’m managing my blood sugar. And actually, I’m not sure if the guy can get a tight shot of that. But this just popped up and it will appear in the cloud. It’s warning me that I’m now going high. So in a second the website is also going to announce, and I’ll get a notification on my band and on my watch, and people are going to start calling me and it will be scary. So there we go. Notification just showed up. So let’s talk about how that notification did, in fact, show up. I’ve got a band. And this is some data in the cloud. How do I see that? Well, turns out the band can talk some JavaScript.

I can take JavaScript out of the band. This is at developer.microsoft.band.com. You may have a band. You may not be a developer, but you may have a URL that points to JavaScript or a blog or some XML and you want to create a tile. I made one for my blood sugar. So now I’ve got Nightscout on my system, and I can see my blood sugar right there. What’s interesting about this is that I can use this Visual Designer, but I can also look at the manifest directly and look at this. That’s the code there. Sugar is high. If it’s over 150, notification. And then I’ll get a notification on my band. This could be done with a build server, whether you want to water your plants, whatever.

That is the personal Internet of Things. And that’s what makes me happy. But let’s think about this. Once my blood sugar is in the cloud, what else can I do with it? I can send notifications. My wife can be concerned. She’s probably going to be calling in a moment to make sure that I’m okay. But this is part of the process. Remember how I said I want to analyze this stuff in Excel. Well, it turns out that I’m not very good at VBA, Visual Basic for Applications. Remember that 80 percent of the world’s business logic runs in Excel. I think that’s a fact. And in fact, there’s an Excel JS API, JavaScript API for Excel. I can write an add-in for Excel in JavaScript using the Web technologies that I know how to make. So we went and we made the HanselSugarsProject, and I’m going to actually show you some of the code here in just a second.

There we go. Where we’re going to populate a table in Excel using JavaScript, using Web technologies. And what’s exciting for this, exciting for me about this, is that I didn’t know how to do this before. You see? I’m a Web developer. But now I can go in here put in my target range, hit refresh, and now we’re going to go and talk to the backend system and then dynamically populate a chart in Excel of my blood sugar that I could then send to a doctor or my wife.

You have to think about the sense of power and enthusiasm I had suddenly have for Excel that I did not have before. I’m a Web developer. That’s a Web application in the pane there. And it works in Excel Online. And one day it will work in Excel on an iPad. It’s like PFT. Once you learn how to use one Lego piece, you can then plug them into other stuff. It’s extremely heartening for me to do this. Again, if my blood sugar isn’t interesting to you, and it shouldn’t be, maybe some hobby of yours or something at work or some inspiration will come into your mind and you’ll think about how you can plug these things in together. That’s what’s so exciting about this. So I’ve got all my blood sugar in real time. I can collect it into the cloud.

What else can I do with it? Let’s switch over to another machine here. I’m going to switch over to a Mac. This is a Mac running VS Code. And this particular Mac has got the code to collect my data from the whoops from the health API. So we’re going to go and make RESTful API calls, standard API calls off to Microsoft Health, which is where the Band-Aid is stored because this is the thing that’s so important.

It’s my data. It’s my heartbeat data. My blood sugar is my data. Once I have that ownership of that data, what can I do with it? I can plug in that data and do different stuff. Let me see if I can jump out of here. This has gone full screen. Full screen can be a little confusing. There we go. I’m going to take that data, and I’m going to pull it out of the Microsoft Health API. And this is a storage explorer. The reason I wanted to show you this on a Mac is because this is the Azure Storage Explorer that we’ve released. That’s cross-platform. We’re looking at the CSV files of my blood sugar data as it’s sitting in the cloud. Excuse me. My heart rate data, pardon me. I’m going to hit download. I’m going to throw that on a desktop. Watch the right side. Now I have power. Now I have control. Now I’ve got a hold of that and I can run analysis on it. And now, in this case, I just brought it down to the desktop.

I can do some test notifications on my blood sugar. I’ve got I can do some analysis here, but maybe I want to do that analysis in a more sophisticated way. When you start thinking about the Internet of Things, you start thinking about huge amounts of data. I said it has my heart rate. Like how much is that? Is it an average over a minute, over five minutes, over hours? This is a huge amount of data. My blood sugar is doing three or 400 datasets every single day. Over the course of a year, this adds up to huge amounts of information. This is where simply putting it into Excel and sorting going looks like a pattern isn’t going to be enough for me. I’m going to need to do some machine learning. I want to find out about why does my blood sugar go up when I’m stressed out.

What does Thanksgiving dinner look like? Maybe I can answer those questions with machine learning. So I’m going to switch back over to this system here. And what I’m going to do, I’m going to jump into here. This is a machine learning kind of algorithm that I’ve put together, experiment, if you will, to pull in data from the heart rate on the right there from Azure Storage and the glucose data and I’m going to come up with a stress index. I’m inventing a number the Hanselman Stress Index. And we’re going to load that data up. And what we’re going to do is clean it up, do a little bit of normalization, do a couple of analysis, run some statistics, and then come up with a dataset that the result is going to be a time-based dataset of my stress.

Okay. So that’s kind of interesting. It’s pretty powerful, though, that but I need to see it over time. So how would I express this over time? And what value is there in it? Well, there is this API in Office 365 and all of Microsoft called the Graph API, the Microsoft Graph, graph.Microsoft.com, and this allows me to pull my information. I keep stressing, it’s my blood sugar, my heart rate, and my calendar and my email. I want a unified API to go and get those things and the graph API, you can see there at graph.Microsoft.com allows me to make those requests.

So what if we looked at my blood sugar and my stress and my heart rate over time, overlaid on an Office 365 calendar, that could give us some insights. And again I don’t know anything about this stuff. This is what’s so important and why as a developer this is so exciting. I know JavaScript. I know the Web. I know C#. Now I’m feeling comfortable about the cloud. But when I see, hey, this is an API and it uses JavaScript and I can access all the graph data in Microsoft, that’s amazing. That’s a playground of excitement. So I can go build this, get a start date and end date. This is all standard parsing of JSON, parsing of JavaScript, all using the graph API and let’s see what kind of question that could potentially answer. So here is my calendar; the stress index as it applies to calendar data. Let’s click on one. This is all real. So here’s glucose is updating. I can see the different rehearsals and I can see heart rate and sugar and stress changes. I can actually run queries and see the result of that machine learning and ask the really burning question, the question that we all have, which is what is the most stressful person in my life that is causing these problems and raising my blood sugar.

[Laughter] [Applause] Darn it. It is a really, really great time to be a developer. I can put stuff like this together now. It is an extremely personal time. It’s an extremely empowering time. You know how we keep telling everyone: Teach the kids how to code; everyone needs to learn how to code. That’s a part of it. But we need to teach people how to think about systems and how to plug stuff in. I don’t know the Graph API. But it made sense because it’s using open Web technologies. I don’t know a lot of open source systems.

But I’m learning those libraries. And I’m bringing them in because they’re using open tech.NET is open source. If I have questions, I can look at the docs but I can also read the source. Maybe I want to run a Mac. Maybe I want an iPhone. I can build those systems with the mobile tools in Visual Studio. Maybe I want to run on Ubuntu then I can use Visual Studio code and I can write Angular and use Go and run those things in the Azure cloud. It’s such an exciting, empowering time to be a developer now. I really hope you have as much fun building stuff as we have had to build this stuff for you. And I really look forward to seeing what you all can build. Thank you very much and have a really great day.