Niels Berglund
Technology musings about coding and data. Some topics: .NET, SQL Server, Data Science, R, Windows Azure and a lot more.
© 2019 nielsb. Some Rights Reserved.
Technology musings about coding and data. Some topics: .NET, SQL Server, Data Science, R, Windows Azure and a lot more.
© 2019 nielsb. Some Rights Reserved.
Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.
As many of you know, I am an un-ashamed .NET developer on the Windows platform. Seeing that I do a lot of SQL Server development my development and OS platforms of choice is sufficient for my work, as I on my development box (Windows of course) install SQL Server Developer edition and whatever .NET framework I use.
That has been sufficient up until now when I want to develop against Kafka. At Derivco we are getting more and more interested in Kafka, and obviously, I want to install it so I can play around investigate it. However, to install it on my development machine as I would typically do with technologies I am interested in, SQL Server, RabbitMQ, etc., is difficult, if not impossible.
NOTE: Yes I know, there are articles on the web discussing how to run Kafka on Windows, but it is a hit and miss whether it works.
A while ago I wrote a post about how to run Kafka under Windows Subsystem for Linux (WSL), and yes it works, but I have had issues with it, and to me, it is still a hack. So the options then (if we rule out WSL) are:
The rest of this post goes through how to set up the Confluent Platform in a Docker environment and use it from .NET.
Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.
Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.
A little while ago I wrote a blog post, Java & SQL Server 2019 Extensibility Framework: The Sequel, about changes in SQL Server 2019 CTP 2.5 impacting how we write Java code for use from SQL Server. While I wrote that post, Microsoft released SQL Server 2019 CTP 3.0, and, (surprise, surprise), that release contains more changes affecting Java code in SQL Server.
This post covers those changes as well as discusses what SQL Server Extensibility Framework and Language Extensions are.
Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.
Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.
As you may know, a while back I wrote some posts about the support for Java in SQL Server 2019: SQL Server 2019 Extensibility Framework & Java. The posts covered in some detail how Java in SQL Server worked, and how to write Java code for use in SQL Server. However, a week or two ago “the sky came tumbling down” when Microsoft released SQL Server 2019 CTP 2.5.
NOTE: CTP stands for Community Technology Preview and is like a beta release.
What Microsoft did in CTP 2.5 was to introduce Java Language Extension SDK, and your Java code now needs to inherit an abstract base class from the SDK. This requirement makes a large part of my previous posts “null and void”, so in this post, we look at what to do going forward.
Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.
Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.
Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.
Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.
Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.
This post is the fourth in a series about installing R packages in SQL Server Machine Learning Services (SQL Server ML Services). To see all posts in the series go to Install R Packages in SQL Server ML Services Series.
Why this series came about is a colleague of mine Dane pinged me and asked if I had any advice as he had issues installing an R package into one of their SQL Server instances. I tried to help him and then thought it would make a good topic for a blog post. Of course, at that time I didn’t think it would be more posts than one, but here we are.
In this post, we look at:
CREATE EXTERNAL LIBRARY requires.Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.
This post is the third in a series about installing R packages in SQL Server Machine Learning Services (SQL Server ML Services). To see all posts in the series go to Install R Packages in SQL Server ML Services Series.
Why this series came about is a colleague of mine Dane pinged me and asked if I had any advice as he had issues installing an R package into one of their SQL Server instances. I tried to help him and then thought it would make a good topic for a blog post. Of course, at that time I didn’t think it would be more posts than one, but here we are.
In this post, we look at how we can use T-SQL and DDL commands to install packages in a remote SQL Server.
Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.
Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.
Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.
Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.
This post is part of the SQL Server 2019 Extensibility Framework & Java series of posts, and it is the second post discussing SQL Server 2019, Java and the creation and use of external libraries.
In the previous post about external libraries, we said that they were beneficial as they reduced complexities when deploying code, but there were still some caveats. So, in this post, we look at how to overcome those caveats
Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.
A couple of months ago I wrote a series of posts about one of the new features in SQL Server 2019; the ability to call out to Java code from inside SQL Server.
To see the posts, go to SQL Server 2019 Extensibility Framework & Java.
In the posts, we discussed how the Java extension differs from R and Python in that R and Python are an integrated part of the SQL Server install (when enabling in-database analytics), but Java is not. In other words, the use of the Java extension requires Java to be installed beforehand, and this then has implications on permissions. We also discussed how Java is a compiled language, and we execute against a method in a class, whereas with R and Python we send a script to the external engine. The consequence of this is that when we execute Java code, we need to indicate where the compiled code resides, and those locations need specific permissions.
All this creates a level of complexity, and it would potentially be easier if we load the Java code from a well-known place, where we do not need to worry about permissions and so forth.
This post is the first of a couple where we see how new functionality in SQL Server 2019 CTP 2.3 can help.
Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.
Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.
Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.
Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.
Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.
Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.
Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.
As you know, SQL Server 2019 introduces the Java language extensions as part of the SQL Server Extensibility Framework. The Java language extensions make it possible to execute Java code from inside SQL Server, the same way we can run R/Python code. Seeing that I am “somewhat” interested in the SQL Server Extensibility Framework I wrote some posts about the Java language extensions: SQL Server 2019 Extensibility Framework & Java.
The code in the posts is very, very simplistic where I was just trying to get the ideas across how and what to do. Another reason the code is simplistic is that I am not a Java “person”; in fact, the only Java code I have ever written is what is in those posts. Me not being a Java “person” I do not have a Java IDE, (for the code I wrote I did not need one either), so I used my regular text editor Sublime Text 3, and I compiled the code from command line ($ javac ...).
I then started on a new Java post (which I have not finished yet), and it became clear I needed an IDE, (intelli-sense, debug, etc.), but which IDE?
Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.
Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This is the “roundup” of the posts that has been most interesting to me over the Christmas and New Year period.
This post is the fourth post in a series where I look at the Java extension in SQL Server, i.e. the ability to execute Java code from inside SQL Server.
To see what other posts there are in the series, go to SQL Server 2019 Extensibility Framework & Java.
This fourth post acts as a “roundup” of miscellaneous “stuff” I did not cover in the three previous posts, and some of the things we look at are:
CLASSPATH environment variable.Ok, let us get into it.
This post came about due to a question on the Microsoft Machine Learning Server forum. The question was if there are any plans by Microsoft to support more the one input dataset (@input_data_1) in sp_execute_external_script. My immediate reaction was that if you want more than one dataset, you can always connect from the script back into the database, and retrieve data.
However, the poster was well aware of that, but due to certain reasons he did not want to do it that way - he wanted to push in the data, fair enough. When I read this, I seemed to remember something from a while ago, where, instead of retrieving data from inside the script, they pushed in the data, serialized it as an output parameter and then used the binary representation as in input parameter (yeah - this sounds confusing, but bear with me). I did some research (read Googling), and found this StackOverflow question, and answer. So for future questions, and for me to remember, I decided to write a blog post about it.
This post is the third post in a series where I look at the Java extension in SQL Server, i.e. the ability to execute Java code from inside SQL Server.
To see what other posts there are in the series, go to SQL Server 2019 Extensibility Framework & Java.
In this post we look at something related to the data passing post; how to handle null values.
Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.
NOTE: I started with these roundup posts in the beginning of 2017 as a way to force myself to write, and - so far, so good. It is now coming up on Christmas and New Year, and I will take a break with these posts and come back in the beginning of next year.
Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.
This post is the second post in a series about SQL Server 2019 Extensibility Framework and the Java language extensions. In the first post, SQL Server 2019 Extensibility Framework & Java - Hello World, we looked at how to install and enable the Java language extensions, and we also wrote some pretty basic Java code to ensure it all worked.
To see what other posts there are in the series, go to SQL Server 2019 Extensibility Framework & Java.
In this post we look at how we can pass data back and forth between SQL Server and Java.
Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.
The SQL Server 2016 release introduced the SQL Server Extensibility Framework (EF), which gives us the ability to, from inside SQL Server, execute code in an external language runtime environment. SQL Server 2016 supports R as external runtime, and Microsoft added Python to supported runtimes in the SQL Server 2017 release. The important part about the EF is that the runtime is outside of the core database engine, but we call it from inside SQL Server via the stored procedure sp_execute_external_script. We can push data from SQL Server queries to the external runtime, and consume data, (resultsets, output parameters) from the external runtime back in SQL Server.
NOTE: You can read more about the actual implementation of the external runtimes and
sp_execute_external_scriptin my SQL Server R Services Series posts.
In SQL Server 2019 Microsoft added the ability to execute custom Java code along the same lines we execute R and Python, and this blog post intends to give an introduction of how to install and enable the Java extension, as well as execute some very basic Java code. In future posts, I drill down how to pass data back and forth between SQL Server and Java.
To see what other posts there are in the series, go to SQL Server 2019 Extensibility Framework & Java.
Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.
Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.
Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.
At the Microsoft Ignite 2018 conference back in September Microsoft released SQL Server 2019 for public preview, and I wrote two short blog posts about it:
What Microsoft also announced was SQL Server 2019 Big Data Clusters, which combines the SQL Server database engine, Spark, and HDFS into a unified data platform! Yes, you read that right: SQL Server, Spark, and Hadoop right out of the box. Seeing that both Spark and Hadoop are mainly Linux based, what makes the Big Data Cluster possible is SQL Server on Linux. When you deploy a SQL Server 2019 Big Data Cluster, you deploy it as containers on Kubernetes, where the Kubernetes cluster can be in the cloud, such as Azure Kubernetes Service, or on-prem like Red Hat OpenShift or even on a local dev-box/laptop using Minikube.
Initially, this post was about SQL Server 2019 Big Data Clusters on Minikube, but after quite a few failed installation attempts I realised I did not have enough memory on my development box, so I decided to try it on Azure Kubernetes Service (AKS) instead.
NOTE: If you want to run SQL Server 2019 Big Data Clusters on Minikube it is suggested that your host machine (Minikube is essentially a VM) has at least 32Gb of memory, and you allocate at least 22Gb to the Minikube VM.
Since I am a complete novice when it comes to Kubernetes, this post covers both how I set up AKS as well as the deployment of SQL Server 2019 Big Data Clusters to AKS, and the post is somewhat a summary of the official documentation.
Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.
Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.
Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.
Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.
Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.
Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.
By the time I publish this blog post MS Ignite is over. During Ignite, Microsoft announced quite a few new things, amongst them SQL Server 2019 with a whole lot of new features and functionality.
I touched briefly on some of them in my What is New in SQL Server 2019 Public Preview post. A couple of things that caught my eye were that SQL Server 2019 for Linux now supports In-Database analytics, what we know as SQL Server Machine Learning Services (R and Python), as well as the Java language extension.
NOTE: You may ask yourself what the Java language extension is; well, that is the ability to access Java code from T-SQL. It is a little bit like SQLCLR, but it executes outside of the SQL Server memory and process space.
Seeing that I never really have played around with SQL Server for Linux, mostly due to that in previous versions (2017) it did not have support for In-Database analytics, I thought that now would be a good time to have a look.
If you read my roundup for week 38, which I published yesterday, you probably noticed that MS Ignite started today. I mentioned in the post that I was particularly interested in some of the SQL Server sessions, as they looked very interesting.
However, even before the sessions started, Microsoft released SQL Server 2019 CTP 2.0 for public preview and, naturally, I jumped on the download link and started downloading. I managed to get to the link in time before the rest of the world started the download, so I managed to get it down and then did an install.
The rest of this post is about my initial findings mostly in the SQL Server Machine Learning Services space.
NOTE: I have looked at SQL Server 2019 the grand total of an hour, so this is a short post.
Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.
Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.
Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.
Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.
Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.
Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.
In my Goodbye Jekyll, Welcome Hugo! post I wrote how I have changed blog engine from Jekyll to Hugo, and how everything seemed to work. Saying that obviously jinxed it, as after I deployed to the hosting site my Atom feed was completely hosed.
I have now spent 4 hours trying to fix everything, but I am not sure I have succeeded. So this is a test post to see if things look OK.
Back in 2013 I wrote a post about how I moved from a self hosted WordPress blog to a static blog; OctoPress. OctoPress spoke to my “geeky” side, as the foundation of it is the static blog-generator Jekyll, which in turn is Ruby based. In fact, OctoPress is more or less just some extra Ruby plugins on top of Jekyll, and to generate sites with OctoPress / Ruby you need Ruby installed on you machine.
I thought that by changing the blog platform, I might write more posts just due to the “geekiness” of the blog engine. Fast forward to the end of 2016, and the number of posts I had written since the switch came to a grand total of three. Those three includes the post where I announced the switch. Yeah, switching increased indeed my productivity - NOT!
So what does this have to do with anything?
Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.
Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.
In the Microsoft SQL Server R Services - sp_execute_external_script - III post I wrote about sp_execute_external_script (SPEES) and the SQL Server Compute Context (SQLCC). Afterwards I realised I had some things wrong, so I wrote a followup post: sp_execute_external_script and SQL Compute Context - I where I tried to correct my mistakes from the initial post. That post led to sp_execute_external_script and SQL Compute Context - II and now we have a mini-series.
To see other posts (including this) in the series, go to sp_execute_external_script and SQL Server Compute Context.
In the previous post in this series, we looked at how data is sent to the SqlSatellite from SQL Server when we are in the SQLCC. This post was meant to look at what goes on inside SQL Server when we execute in SQLCC, but I realised that it would make more sense if, before we look at the internal working when in SQLCC, I covered what happens when pulling data in the local context. So that is what this post is all about.
Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.
Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending. Unfortunately there was not much that caught my eye this week, so this is a short post.
Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.
You who follows my blog and have read my weekly roundups you know that I am quite (that is an understatement) interested in Apache Kafka and I am curious to find out what “cool” things one can do with it. For that, I want to be able to test “stuff” quickly. When I test and try out new things, I usually do it on my development box which contains everything I need: SQL Server, RabbitMQ, RStudio, Microsoft Machine Learning Server, Visual Studio and the list goes on.
So seeing that I have most of my “tools of the trade” on my machine I obviously also would like to have Kafka on the box. Herein lies a problem, I am a Windows dude and Kafka, and Windows do not gel. Yes, some people are running Kafka on Windows, but it is a chore. OK, so what to do? Sure, I could potentially run Kafka on a virtual machine, or in a Docker image, but it is not as transparent as I would like it to be (yeah, I am lazy).
Hmm, Microsoft did introduce the ability to run Linux binary executables (in ELF format) natively on Windows 10 in Windows 10 version 1607. The feature is called Windows Subsystem for Linux (WSL), and since I am now running version 1803, maybe I should try and install Kafka in WSL.
Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.
I wrote the post sp_execute_external_script and SQL Compute Context - I about how the SQL Server Compute Context (SQLCC) works with sp_execute_external_script (SPEES), as I wanted to correct some mistakes I did in the Microsoft SQL Server R Services - sp_execute_external_script - III post. I initially thought one post would be enough, but quite soon I realised I was too optimistic, and at least one more post would be needed, if not more. So this is the first followup post about SPEES and SQLCC.
To see other posts (including this) in the series, go to sp_execute_external_script and SQL Server Compute Context.
One of the reasons for me realising that one post is not enough is that while I wrote and executed code for the first post, I noticed some fairly significant performance differences using SQLCC compared to not using SQLCC (SQLCC performed better :)). So that is part of what we look at in this post.
Geez, does time fly or what? We are already past the halfway mark of the year! Anyway, throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.
This post is the second post in a series about installing R packages in SQL Server Machine Learning Services.
To see all posts in the series go to Install R Packages in SQL Server ML Services Series.
Why this series came about is a colleague of mine Dane pinged me and asked if I had any advice as he had issues installing an R package into one of their SQL Server instances. I tried to help him and then thought it would make a good topic for a blog post. Of course, at that time I didn’t think it would be more posts than one, but here we are.
Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.
This post will (hopefully) be short and sweet. It came about as I was testing out “stuff” for the Install R Packages in SQL Server ML Services series of posts and I could not get it to work as I had expected.
This post was supposed to be a single post about how to install R packages in SQL Server Machine Learning Services, but once again I completely misjudged the scope of the topic. So this one post turned into a series of posts about how to install R packages in SQL Server Machine Learning Services and this is the first post in the series.
To see other posts in the series go to Install R Packages in SQL Server ML Services Series.
As you may know, I am in the process of writing the follow-up post to sp_execute_external_script and SQL Compute Context - I. However, I have a hard time getting into the flow of things, so I use any excuse I can, to not have to write. So when Dane Bax, a colleague of mine, contacted me a couple of days ago with a SQL Server Machine Learning Services problem, I jumped at the chance to help him, and also write a post about it.
Dane works for a sister company of Derivco as a data scientist and at both Derivco and the sister company - Microgaming we are now looking at using SQL Server Machine Learning Services.
His problem was that he wanted to use a CRAN package - bsts - which is not part of a standard SQL Server R installation. He tried a couple of things to get it installed but got errors, so he decided to get in touch with me.
Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.
Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.
Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.
This post is as much for you as it is for me to come back to when I wonder what to do when SQL Serer Machine Learning Services all of a sudden stops working after a SQL Server upgrade.
Last week I did a complete reformat of my development box at home and installed the latest version of Windows 10: the 1803 build, also called the April 2018 update.
NOTE: I usually try to reformat my box at least twice a year, to ensure it is not too “bogged” down by old “cruft”, and that it has the latest and greatest.
After the reformat I installed all my necessary applications, among them SQL Server 2017 with the in-database analytics components (R and Python). After installation I tested that it all worked and made sure I could execute my usual test scripts for sp_execute_external_script:
EXEC sp_execute_external_script @language = N'R'
, @script = N'd <- 42'
Code Snippet 1: Test Script for R
The “moral” equivalent to Code Snippet 1 in Python also worked without an issue:
EXEC sp_execute_external_script @language = N'Python'
, @script = N'd = 42'
Code Snippet 2: Test Script for Python
Cool, all is “A-OK”! A couple of days go by, and I see that there is a Cumulative Update (CU) for SQL Server 2017 - CU7. I install it and does not think much about it. I mean: “what can go wrong, how hard can it be?”. A couple of days later and I am busy writing the follow-up post to sp_execute_external_script and SQL Compute Context - I when I try to execute sp_execute_external_script, and it falls over!
Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.
Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.
The last blog post in the Microsoft SQL Server R Services series covered some performance related parameters of sp_execute_external_script, (@parallel and @r_rowsPerRead), and it touched a little bit on SQL Server Compute Context.
After I published the post, I received an email from Umachandar Jayachandran who is a Principal Program Manager in the SQL Server team and is working on integration of R, Python and other runtimes into SQL Server (basically, he knows this “stuff”). In the mail, he said that I was somewhat incorrect in what I said about how the SQL Server Compute Context (SQLCC) works.
So I decided to write a post to correct my mistakes, and this is it. While I wrote this post I realized that there are more to the SQL compute context that meets ones eye, so I will - in all likelihood - write more posts about SQLCC.
To see other posts (including this) in the series, go to sp_execute_external_script and SQL Server Compute Context.
Before I begin I want to say a very big THANK YOU to Umachandar as well as Nellie Gustafsson who also is a SQL Server Program Manager at Microsoft. Without them making me see the “errors of my way” I would not have been able to write this post.
Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.
Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.
Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.
Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.
Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.
Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.
Back in the day when I started with the SQL Server R Services - Internals posts, I thought I would use SQL Server Extended Events (XEvents) to figure things out. I knew SQL Server and the extensibility architecture supported XEvents because I had come across this documentation, where it (amongst other things) said:
“SQL Server provides a set of extended events to use in troubleshooting operations related to the SQL Server Trusted Launchpad, as well as Python or R jobs sent to SQL Server.“
From the above, it is pretty clear that XEvents is the way to go when drilling down into how things “hang together”. However, how hard I tried, I could not make “head or tails” of how to get things to work (I should probably have tried harder), so in the end, I went the WinDbg and WireShark route.
Fast forward a year (and a bit) and someone asked a question on an internal forum about XEvents and SQL Server Machine Learning Services, and I decided to look into it and to try to understand it a bit better. So this post is the result of me playing around with researching XEvents and SQL Server Machine Learning Services.
Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.
Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.
This post is the 24:th post about Microsoft SQL Server R Services, and the third that covers sp_execute_external_script. I have thought many times that this series has reached its end, and I have many times realised I was wrong. Well, now I can say that this is the last post in the series! I started over a year ago (March 4 2017) with an idea about “wouldn’t it be cool if I wrote a couple of posts about Microsoft SQL Server R Services”! Over one year and 24 posts later, here we are!
This last post talks about sp_execute_external_script and a parameter named @parallel. When we discuss this parameter, we also touch (a little bit) upon performance aspects of sp_execute_external_script.
Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.
Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.
This post is the 23:rd post about Microsoft SQL Server R Services, and a continuation of sp_execute_external_script - I. To see other posts (including this) in the series, go to SQL Server R Services.
I honestly thought this post would be the last in the series (I have thought quite a few times), but alas, that is not the case. There will be at least one more post about sp_execute_external_script after this.
This is the 22:nd post in a series about Microsoft SQL Server R Services. This journey started about a year ago when I thought it would be a good idea to write a couple of blog posts about, what then went under the name of, Microsoft SQL Server 2016 R Services. Never in my wildest fantasies did I think I would write over 20 posts.
NOTE: Since the introduction of Microsoft SQL Server R Services, it has been renamed to Microsoft SQL Server Machine Learning Services.
To see other posts (including this) in the series, go to SQL Server R Services.
This series began with a discussion about how to install Microsoft SQL Server R Services and continued with the internals of how it works internally in SQL Server and the launchpad service. Throughout the series, we have used sp_execute_external_script when investigating what goes on. So, I thought that having a couple of posts looking at sp_execute_external_script would be a good way to finish this series off, and this post is the first of three about sp_execute_external_script.
Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.
Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.
Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.
Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.
Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.
This is the 21:th post in a series about Microsoft SQL Server R Services, (it is now called Microsoft SQL Server Machine Learning Services), and the 20:th post that drills down into the internal of how it works.
Wow, 20 posts about internals - when I started this Microsoft SQL Server R Services series, I thought I might achieve five or six posts. I would never have - in my wildest fantasies - thought I would reach twenty!
To see other posts (including this) in the series, go to SQL Server R Services.
We are however now coming closer to the end of this whole series, and this post is the last for Internals. After this post, I foresee one, possibly two, more posts about other things related to Microsoft SQL Server R Services - and then we are done!
Throughout the week, I read a lot of blog-posts, articles, and so forth, that has to do with things that interest me:
This blog-post is the “roundup” of the things that have been most interesting to me, for the week just ending.
Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me
This is the “roundup” of the posts that has been most interesting to me, for the week just gone by.
This is the 20:th post in a series about Microsoft SQL Server R Services, and the 19:th post that drills down into the internal of how it works. To see other posts (including this) in the series, go to SQL Server R Services.
We are nearing the end of the Internals series, and if we look at what we have covered, we have almost come full circle.
Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me
This is the “roundup” of the posts that has been most interesting to me, for the week just gone by.
This is the 19:th post in a series about Microsoft SQL Server R Services, and the 18:th post that drills down into the internal of how it works. To see other posts (including this) in the series, go to SQL Server R Services.
In this post we’ll continue looking at data coming back to SQL Server, and what happens code-wise in SQL.
Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me:
This is the “roundup” of the posts that has been most interesting to me over the Christmas and New Year period.
This is the 18:th post in a series about Microsoft SQL Server R Services, and the 17:th post that drills down into the internal of how it works. To see other posts (including this) in the series, go to SQL Server R Services.
The last Internals posts have discussed the transport of data between SQL Server and the SqlSatellite. In Internals - XVI we looked at how data came back from the SqlSatellite, and what the data looked like.
This is the 17:th post in a series about Microsoft SQL Server R Services, and the 16:th post that drills down into the internal of how it works. To see other posts (including this) in the series, go to SQL Server R Services.
In the last few posts in this Internals series we have looked at how data is being transported from SQL Server to the SqlSatellite. In this post we’ll look at how data is going the other way, from the SqlSatellite to SQL Server.
Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me
This is the “roundup” of the posts that has been most interesting to me, for the week just gone by.
NOTE: I started the “roundup” posts in the beginning of this year, as a way for me to force myself to keep my blog active, and to share with you what I find interesting in the tech-world. It is now coming up on Christmas and New Year, and I will take a break with these posts and come back in the beginning of next year.
Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me
This is the “roundup” of the posts that has been most interesting to me, for the week just gone by.
NOTE: This week there is very little content. This is because I’ve been overseas - last Sunday I went to Gibraltar, and I came back yesterday (Saturday) evening. Gibraltar is an awesome place by the way. The downside of living in South Africa (as I do) is the travel time - Durban to Gib was a 25 hours door to door trip!
Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me
This is the “roundup” of the posts that has been most interesting to me, for the week just gone by.
This is the 16:th post in a series about Microsoft SQL Server R Services, and the 15:th post that drills down into the internal of how it works. To see other posts (including this) in the series, go to SQL Server R Services.
In the last post Internals - XIV we looked at the Binary eXchange Protocol (BXL). The post was fairly long, and there were things that I wanted to cover which I didn’t do due to the length of the post.
So this post is a continuation of Internals - XIV.
Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me
This is the “roundup” of the posts that has been most interesting to me, for the week just gone by.
This is the 15:th post in a series about Microsoft SQL Server R Services, and the 14:th post that drills down into the internal of how it works. To see other posts (including this) in the series, go to SQL Server R Services.
Finally, after a couple of false starts this post will look into the protocol used when transferring data back and forth between SQL Server and the SqlSatellite: Binary eXchange Language (BXL). Why write about it at all? Well, the big reason why I wanted to know more about it, was due to the strange packet sizes I saw when inspecting data being sent from SQL Server to the SqlSatellite: one row with 5 numeric columns had a packet size of 6300 bytes!!!
Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me
This is the “roundup” of the posts that has been most interesting to me, for the week just gone by.
Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me
This is the “roundup” of the posts that has been most interesting to me, for the week just gone by.
This is the fourteenth post in a series about Microsoft SQL Server R Services, and the thirteenth post that drills down into the internal of how it works. To see other posts (including this) in the series, go to SQL Server R Services.
Sorry guys, in my last post Internals - XII I said we’d - in this post - look into the Binary eXchange Language protocol(BXL) which is used when sending data from SQL Server to the SqlSatellite. The protocol is also involved when we send one row of data containing 5 integers, and the packet size for that ends up being 6300 bytes.
Well, we won’t look at that in this post, I was once again led astray when researching how data is sent.
Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me
This is the “roundup” of the posts that has been most interesting to me, for the week just gone by.
This is the 13:th post in a series about Microsoft SQL Server R Services, and the 12:th post that drills down into the internal of how it works. To see other posts (including this) in the series, go to SQL Server R Services.
The last few posts in this series has been about communication between SQL Server and SqlSatellite. We have discussed the connection between the two, and we have also seen how packets are sent from SQL Server to SqlSatellite. In this post we’ll continue looking at what causes the packets to be send and what sends them. We will specifically look at the packet containing the external script, as well as the packet that contains the data (@input_data_set_1).
Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me
This is the “roundup” of the posts that has been most interesting to me, for the week just gone by.
Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me
This is the “roundup” of the posts that has been most interesting to me, for the week just gone by.
This week I have very little to post, mainly due to me being out of the country, and not having been able to read as much as I would have liked.
This is the twelfth post in a series about Microsoft SQL Server R Services, and the eleventh post that drills down into the internal of how it works. To see other posts (including this) in the series, go to SQL Server R Services.
In the last post we spoke about how the transfer of data happens between SQL Server and the R components, and we saw how data was transferred over the TCP connection between the SqlSatellite and SQL Server. In the post we mentioned that the protocol being used is Binary eXchange Language protocol(BXL), and this post was supposed to talk about BXL.
Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me
This is the “roundup” of the posts that has been most interesting to me, for the week just gone by. This week, the roundup comes a bit early due to me having to go abroad at the weekend for a week.
Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me
This is the “roundup” of the posts that has been most interesting to me, for the week just gone by.
Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me
This is the “roundup” of the posts that has been most interesting to me, for the week just gone by.
Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me
This is the “roundup” of the posts that has been most interesting to me, for the week just gone by.
Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me
This is the “roundup” of the posts that has been most interesting to me, for the week just gone by.
Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me
This is the “roundup” of the posts that has been most interesting to me, for the week just gone by.
Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me
This is the “roundup” of the posts that has been most interesting to me, for the week just gone by.
I know, I know - there is not much for this week. A major reason for this is that, here in South Africa, we are in SQL Saturday season right now. What has that got to do with anything you may ask - well, it is like this:
This is the eleventh post in a series about Microsoft SQL Server R Services, and the tenth post that drills down into the internal of how it works. To see other posts (including this) in the series, go to SQL Server R Services.
In Internals - IX I said we were nearing the end of the internals part is this series, and I had planned this post to be the final post in the Internals part. However while investigating how data is sent between SQL Server and the external components I realized that I would need to probably do two more posts (apart from this) for Internals.
Anyway, in this post we’ll see how data is sent to the R components from SQL Server.
Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me
This is the “roundup” of the posts that has been most interesting to me, for the week just gone by.
Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me
This is the “roundup” of the posts that has been most interesting to me, for the week just gone by.
This is the tenth post in a series about Microsoft SQL Server R Services, and the ninth post that drills down into the internal of how it works. To see other posts (including this) in the series, go to SQL Server R Services.
We are now nearing the end of the Internals part of this series (you probably say “about time”). When I started this series way back in March this year, I thought that the Internals would be two, maximum three episodes. Boy was I wrong!
In this post we’ll look at how the various components are communicating with each other.
Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me
This is the “roundup” of the posts that has been most interesting to me, for the week just gone by.
Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me:
This is the “roundup” of the posts that has been most interesting to me, for the week just gone by.
Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me
This is the “roundup” of the posts that has been most interesting to me, for the week just gone by.
Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me
This is the “roundup” of the posts that has been most interesting to me, for the week just gone by.
A little while ago I wrote a blog-post about the changes in the SQLCLR security model in SQL Server 2017. I wrote about how Microsoft has changed the relation between CAS and security boundaries, and CAS is no longer supported as a boundary! And also how Microsoft introduced an sp_configure option called clr strict security, which by default is set to 1 (on). When the setting is on, SQL Server treats all assemblies (SAFE, EXTERNAL_ACCESS, UNSAFE) as if they were marked UNSAFE.
The release I wrote the post against was CTP 2.1, and earlier this week Microsoft released SQL Server 2017 RC1. Some days ago I received a comment from Paul Vestuto on my post, pointing out that in RC1 there has been some more changes to the security model. I thought those changes would earn their own blog-post, instead of just editing the previous post.
This is the ninth post in a series about Microsoft SQL Server R Services, and the eight post that drills down into the internal of how it works. To see other posts (including this) in the series, go to: SQL Server R Services
So far in this series we have been looking at what happens in SQL Server as well as the launchpad service when we execute sp_execute_external_script, and we have still no real “clue” to where the R engine comes into play.
Well, hopefully that will change (at least a little bit) with this post, as we here will look at what happens when we leave the launchpad service.
Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me
This is the “roundup” of the posts that has been most interesting to me, for the week just gone by.
(WOW, I finally managed to get this post out! Yay to me!!)
This is the eighth post in a series about Microsoft SQL Server R Services, and the seventh post that drills down into the internal of how it works. To see other posts (including this) in the series, go to SQL Server R Services.
In this post we will look more at some of the directories that are created when we execute sp_execute_external_script, as well as files being created in those directories.
Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me
This is the “roundup” of the posts that has been most interesting to me, for the week just gone by.
Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me
This is the “roundup” of the posts that has been most interesting to me, for the week just gone by.
While I was writing the post about SQLCLR and Certificates, I came across some strange behavior when I accidentally ran my code on an SQL Server 2017 instance. I quickly connected back to my SQL Server 2016 instance and continued. However, afterwards I started to look into the differences between SQLCLR in SQL Server 2017 and SQL Server 2016, and this post is the result of me playing around investigating the changes.
This post originates, like the one about sqlrutils, from a question on a Microsoft forum - this time the .NET Framework inside SQL Server forum. As the name implies, the forum cover questions about the usage of SQLCLR (.NET inside SQL Server), and the question was about signing of SQLCLR assemblies in order to assigning other permission sets than SAFE to the assembly.
As I, back in the day, taught SQLCLR as part of Developmentor’s SQL Server curriculum, I thought that it’d be interesting if I tried to answer the question, and also make a blog-post about it. That way I’d remember what to do the next time the question came up.
NOTE: In another post, I wrote this about Developmentor: Developmentor were back in the day THE training company to go to if you wanted highly, highly technical training about COM, .NET, SQL Server, etc. This article by my old colleague Ted Neward (@tedneward) sums DM up quite accurately (apart from the fact that DM hadn’t closed its doors when the article was written, ooops). Ted was probably more accurate than he at the time thought, since DM went out of business somewhat later.
Anyway, let’s see what this is all about…
Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me
This is the “roundup” of the posts that has been most interesting to me, for the week just gone by.
In the Microsoft R Server forum the other day was a question about inputs to SQL Server R stored procedures, or rather about “strange” parameter naming, when using the sqlrutils package. I got intrigued by the question and started playing around with researching it, and this blog-post is the result. Oh, and this blog-post would never have seen the light of the day, if it hadn’t been for the original poster of the question: JD Long, thanks JD!
So, what is SQL Server R stored procedures and what is sqlrutils?
Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me
This is the “roundup” of the posts that has been most interesting to me, for the week just gone by.
Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me
This is the “roundup” of the posts that has been most interesting to me, for the week just gone by.
Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me
This is the “roundup” of the posts that has been most interesting to me, for the week just gone by.
Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me
This is the “roundup” of the posts that has been most interesting to me, for the week just gone by.
Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me
This is the “roundup” of the posts that has been most interesting to me, for the week just gone by.
Time series data is being used more and more in all industries, and of course we want to be able to forecast and predict future values. There are however a variety of challenges that come with producing a large number of forecasts across a variety of time series.
The Facebook’s data science team has worked with time series forecasting for quite a while, and recently they open-sourced an internal tool for this type of forecasting: Facebook Prophet.
Recently there was a post on the Microsoft R Server forum if it was possible to use Prophet with Microsoft R Server, so I thought I’d test it out. This post covers how to use it, both from a standalone Microsoft R Server as well as SQL Server R Services.
This is the seventh post in a series about Microsoft SQL Server R Services, and the sixth post that drills down into the internal of how it works. To see other posts (including this) in the series, go to SQL Server R Services.
In this post we will look at the directories that are created when we execute sp_execute_external_script.
Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me
This is the “roundup” of the posts that has been most interesting to me, for the week just gone by.
Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me
This is the “roundup” of the posts that has been most interesting to me, for the week just gone by.
This is the sixth post in a series about Microsoft SQL Server R Services, and the fifth post that drills down into the internal of how it works. To see other posts (including this) in the series, go to SQL Server R Services.
In Internals - III and Internals - IV, we looked at how the launchpad service creates RTerm processes when we execute an external script.
The Internals - IV post came about due to an email from Bob Albright (@bob_albright), pointing me to some resources (a blog-post) regarding the processes the launchpad service creates.
In his email Bob also, kindly enough, attached some code. The code highlighted some behavior when creating the RTerm processes, especially around parallelism. In this blog-post we’ll follow up on that and look into the effect parallelism has on process creation.
Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me
This is the “roundup” of the posts that has been most interesting to me, this week.
Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me
This is the “roundup” of the posts that has been most interesting to me, this week.
This is the fifth post in a series about Microsoft SQL Server R Services, and the fourth post that drills down into the internal of how it works. To see other posts (including this) in the series, go to SQL Server R Services.
In Internals - III, I wrote about how the launchpad service creates multiple processes when executing an external script.
Seeing that some of the conclusions I came to was somewhat educated guesses, I asked you guys to correct me where I was in-correct and/or add more information. After that post - Bob Albright (@bob_albright) - wrote me an email and pointed me to some resources around process creation, as well as some demo code. Thanks Bob!
So today we’ll drill even further into the creation of processes, and see how they are used.
On April 19, 2017 Microsoft held an on-line conference Microsoft Data Amp to showcase how Microsoft’s latest innovations put data, analytics and artificial intelligence at the heart of business transformation. The keynotes speakers were Scott “Red Shirt” Guthrie who is an Microsoft Executive Vice President (fairly high on the food-chain), and Joseph Sirosh a Corporate Vice President, head of the Information Management and Machine Learning group. Joseph probably knows a thing or two about data, and Scott - well, Scott knows A LOT!
During the keynotes, Scott and Joseph shared how Microsoft’s latest innovations put data, analytics and artificial intelligence at the heart of business transformation. A big enabler for this is SQL Server 2017 (the “artist” formerly known as SQL Server V.Next), which introduces a lot of very cool new “stuff”.
The keynote speeches were followed by recorded short:ish sessions drilling down into certain aspects of Microsoft’s new offerings. If you are interested in the various presentations at Microsoft Data Amp, they all are on Channel 9.
What caught my eye was the announcement of how SQL Server 2017 now supports Python as an extensible script engine. When Microsoft introduced support for R in SQL Server 2016, rumors immediately surfaced how other engines also would be supported, where Python was high on the list (Julia another one).
Seeing that I am somewhat into SQL Server R Services (the SQL Server 2016 flavor) here, here, here, here, and more posts to come, I really had to have a look.
The rest of this post is a brief introduction to executing Python code in SQL Server 2017, in essence I just want to be able to execute some sort of Python code.
Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me:
This is the “roundup” of the posts that has been most interesting to me, this week.
This is the fourth post in a series about Microsoft SQL Server R Services, and the third post that drills down into the internal of how it works. To see other posts (including this) in the series, go to SQL Server R Services.
In the previous internals posts here and here we have looked at what goes on inside SQL Server when executing an R script, and what happens in the launchpad service when executing.
This post was initially to be about the R runtime, and what other parts are involved when executing R scripts in SQL Server. However, during my “spelunking” around I realized that I did not really understand what happens when we create the processes for the R runtime from the launchpad service and the launcher (which is what I covered in the internals II post).
So, instead of looking at the R runtime and the other R related components, we’ll look at what happens when creating external processes. A subsequent blog-post will cover the R runtime and components.
Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me
This is the “roundup” of the posts that has been most interesting to me, this week.
Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me
This is the “roundup” of the posts that has been most interesting to me, this week.
This is the third post in a series about Microsoft SQL Server R Services, and the second post that drills down into the internal of how it works. To see other posts (including this) in the series, go to SQL Server R Services.
In the previous internals post we looked at what happens inside the SQL Server engine when we execute sp_execute_external_script. In that post we wrapped up when we reached the launchpad service (Launchpad.exe). This post will look closer at the launchpad service, and we will do it by some more “spelunking”
Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me
This is the “roundup” of the posts that has been most interesting to me, this week.
Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me
This is the “roundup” of the posts that has been most interesting to me, this week.
This is the second post in a series about Microsoft SQL Server R Services, and the first post that drills down into the internal of how it works. To see other posts (including this) in the series, go to SQL Server R Services.
In this post, and one or two more we will look at what goes on under the covers when we execute some R code inside SQL Server. This post looks at quite in detail what happens in the SQL engine when we execute sp_execute_external_script.
To try and get an understanding we’ll do something that we did quite a lot back in the day when I worked at Developmentor; we’ll “spelunk” the SQL Server code via WinDbg. This can be really useful when trying to understand and get to grips with new technology/functionality.
NOTE: Developmentor were back in the day THE training company to go to if you wanted highly, highly technical training about COM, .NET, SQL Server, etc. This article by my old colleague Ted Neward (@tedneward) sums DM up quite accurately (apart from the fact that DM hadn’t closed its doors when the article was written, ooops).
Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me
This is the “roundup” of the posts that has been most interesting to me, this week.
Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me:
This is the “roundup” of the posts that has been most interesting to me, this week.
A couple of weeks beck I wrote a blog-post where I compared CRAN R with Microsoft R Server. The comparison was basically how large data sets are handled. In the post I mentioned that SQL Server 2016 (and later) is the delivery mechanism for Microsoft R Server on the Windows platform. In this post we will look at how to install and enable SQL Server R Services.
This post is the first in a series of posts about SQL Server R Services and to see other posts (including this) in the series, go to SQL Server R Services.
Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me
This is the “roundup” of the posts that has been most interesting to me, this week.
Last Saturday (February 18) I was in Cape Town at the second world-wide satRday conference ever, where I gave a talk named: “Microsoft, Open Source, R: You gotta be kidding me!“. The talk was about Microsoft’s R Server offering, and how it in some cases offered a better performance than Open R. Seeing the session lengths were only 20 minutes, I could not show any code, so in a “weak” moment I promised to put the code up on my web-site together with an accompanying blog-post. This is it :).
Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me
This is the “roundup” of the posts that has been most interesting to me, this week.
Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me:
This is the “roundup” of the posts that have been most interesting to me, this week.
A week or two ago I came across a post on the RabbitMQ Users forum, about how to communicate from SQL Server to RabbitMQ. Seeing that we do that a lot at Derivco, I came with some suggestions, and also said that I was writing a blog-post about how it can be done. The part of writing was not entirely correct - at least not until now (sorry Jim, it’s been hectic at work).
Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me:
This is the “roundup” of the posts that has been most interesting to me, this week.
Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me
This is the “roundup” of the posts that has been most interesting to me, this week.
Just to set things straight, the title of this post has nothing to do with US politics, but the infinitely more important (and exciting) subject of transactions in SQL Server and the concept of doomed transactions. Why I came across the idea for this post was due to a discussion I had with a colleague of mine - Sameer Chunilall - who didn’t agree with my post a while ago about XACT_ABORT, he believes it is a good thing (mostly), where I believe it is a bad thing (mostly).
Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me:
This is the “roundup” of the posts that has been most interesting to me, this week.
Throughout the week, I read a lot of blog-posts, articles, etc., that has to do with things that interest me:
This is the “roundup” of the posts that has been most interesting to me. Oh, and if you wonder if you have missed Interesting Stuff - Week #1. seeing that this post is named Interesting Stuff - Week #2, you haven’t. They are numbered by calendar week number, and I started the second week of January :).
SET XACT_ABORT defines whether a transaction is automatically rolled back when a T-SQL statement raises a run-time exception, and when you read posts from prominent SQL bloggers you quite often see that they recommend to always have XACT_ABORT set to ON. From the title of this post you may get the impression that I do not necessarily agree, and to an extent you may be right. So, let us see …
Way back when (in 2010 as a matter of fact), I wrote a couple of blog-posts (here and here) about error handling in the new CTP releases of SQL Server Denali. Denali was what would become SQL Server 2012.
The new functionality built upon what was introduced in SQL Server 2005 - the notion of structured exception handling through BEGIN TRY END TRY followed by BEGIN CATCH END CATCH. In those blog-posts I was fairly positive, and saw the new functionality as something useful and very well worth implementing. I am still positive, however since then I have used the new functionality introduced in SQL Server 2005 extensively in production and have come across some gotchas that I thought would be worth writing a blog-post about.
What can I say, I am crap at this “blogging” thing! Three years ago, I resuscitated my blog, after a hiatus of one year. I also switched blog engine from WordPress to OctoPress (which at that stage was version 2). At the time of the switch, I had had a one year hiatus of blogging (I am not a very prolific blog writer :)), and I remember I was very tempted to write in that post, that if I cannot keep up blogging I will quit. Boy, am I glad I did not write that!
The two first weekends in August I were in Cape Town and Johannesburg respectively presenting at SQL Saturday events. I had a great time, and the audience were terrific. My employer Derivco graciously gave me some “swag” to hand out, and that went down a treat. Thanks Derivco!!
In an earlier post, I wrote how I had changed blog-engine from WordPress to OctoPress. At the moment I am running Windows as my main OS (mostly due to work related requirements), and - as OctoPress is very much Ruby and Python based - there are certain things to be aware of when installing this on Windows. Furthermore, my blog is now hosted on Windows Azure - so, with all this in mind, I thought I’d put together a blog post about the set-up of OctoPress on Windows, running on Windows Azure.
As some of you may know, I - once upon a time - developed a project (VS add-in, templates, etc) for automatic deployment of CLR assemblies to SQL Server: SqlClrProject. That project has been dormant now for a couple of years, but I now and then get requests for where it can be downloaded from (I had it on CodePlex, but had to take it down as I didn’t publish the source code).
I have a while ago just finished watching the live stream of the first keynote (yes there will be one tomorrow as well), at Microsoft BUILD. Having attended / presented, at quite a few of these kind of events - and being somewhat jaded (well OK then, a lot jaded), I must still say that I am impressed.
Transactions in SQL Server seems to be a difficult topic to grasp. This weekend I came across a blog-post where the poster showed a “solution” to the “The ROLLBACK TRANSACTION request has no corresponding BEGIN TRANSACTION” error we sometimes see when various stored procedures call each other. The solution (even though it masked out the error in question) did not get it quite right. So I thought I would make a post about the subject.
This is a first post about my experiences with running F# and Mono on a Mac.
In a previous post I wrote about how I have started to play with F#. As that post also covered SQLCLR it was obvious I was on Windows. Even though I make my living from development in a Windows environment, my main machine is a MacBook, and I run OSX as my main OS. I have previously also been running Linux (ArchLinux) on this machine as my main OS. Naturally I have heard about Mono (and also installed it a couple of times - and quickly un-installed again, but I have not really done anything with it. I have always run Windows in a VM on my MacBook for development etc. However after the announcement that F# was going Open Source, and Tomas P posted about his F# MonoDevelop plug-in, I decided that I should have a look at what it would be like to do F# “stuff in OSX.
At PDC 2010 Microsoft showed the new Async features of coming C# (and VB.NET) versions, and quite a lot has been written about it already. Part of the Async CTP is TPL Dataflow, and this has gone somewhat un-noticed.
Recently I have become very interested in F# and I am at the moment trying to get to grips with it. It is definitely a different beast than C#, but so far I like it - a lot!
Anyway, I am a SQL nerd, and many moons ago I was very heavily involved in SQLCLR (for you who don’t know what that is; it is the ability to run .NET code inside the SQL Server engine. It was first introduced with SQL Server 2005). So I thought it would be a “giggle” to see if I could get some F# code running inside SQL Server.
.. from a relational developers perspective!!
Well, the title may be a bit harsh, but at least it grabbed your attention - did it not?!
A week ago, or so, I wrote a wish list to Santa for Denali from a relational developers perspective. In that wish list I wrote that there has been fairly little love for relational SQL developers in the recent versions of SQL Server, and that I hoped in this version (i.e Denali) Microsoft would “go back to the roots” and give us developers some new stuff.
In my previous post I wrote about the new THROW keyword in Denali / SQL 11. Having played around a bit more with Denali, I wanted to write some additional things about THROW and it’s relation to RAISERROR.
A couple of days ago I wrote my wish-list to Santa what I wanted to see in next version of SQL Server (SQL 11 / Denali). I was pleasantly surprised that I could find out for myself shortly after; i.e. SQL Server Denali CTP1 was released during the PASS Summit. I have literally finished installing the next version of SQL Server (Denali / SQL 11) on a new VM, like 10 minutes ago, and I have done a quick check of the new features of SQL Server Denali (what I could find at least) against my wish-list.
There is probably no secret that Microsoft is working hard on next version of SQL Server. The rumour has it that it will be named SQL 11 (it apparently goes under the code name of Denali. Quiz; MS has used the Denali code name previously, what was it for? Answers in the comments ).
This is a re-post from my previous blog. However, as that blog has now gone to the big blog repository in the sky (or wherever blogs go to when they are no more), I decided to repost this, seeing how CEP and StreamInsight are becoming more and more popular.
Yesterday I downloaded and installed the August CTP of SQL Server 2008 R2, and today I played around with it for a while. So, what are my impressions?
As good as SQL 2005 was (well, still are), one disappointment was that you needed Visual Studio if you wanted to debug your stored procedures. Seriously, what was MS thinking when they did that, especially as in SQL 2000, Query Analyzer had debug capabilities?!!
One of the first thing I usually do after having installed a new version of SQL Server is to snoop around at the system tables to see what new tables there are and what they can give me information about.