The Case of Invoke-RestMethod vs. The Bad JSON Feed

I work with some incredibly smart and talented people on a daily basis at work who build some pretty cool systems that integrate with the services my team runs for our campus.  One of those services is an API that is run by our Identity Management team that gives us an interface to work with all of the identity data for campus.  This REST API allows us to query not only information about accounts in our environment, but feed data back in for things like provisioning email aliases or just notifying them that we’ve given someone a mailbox.  Like I said, pretty cool stuff!

We use the API for both interactively querying account information and for working with accounts in automation.  Several of the members of my team started noticing that some of our PowerShell functions weren’t returning data periodically.  The bells and whistles started going off when we started receiving failures on several pieces of PowerShell automation due to null data being received when pulled data on an individual account.  So, let’s look one piece of code we’re using :

This PowerShell function is a fairly basic example of how to work with a REST API using the Invoke-RestMethod cmdlet.  This particular function lets me run

on myself and get back everything that is stored about my account when Invoke-RestMethod hits the API.  In this case, I’m getting back null data.  That’s not right…

Invoke-RestMethod is a pretty cool cmdlet for working with a REST API.  It’s smart enough to recognize that it is receiving back data from my API in JSON format and take that data and turn it into objects that I can work with in PowerShell.  The small gotcha is that Invoke-RestMethod really depends on this being valid JSON.  See where I’m going here???

So, it looks like we’re getting back null data, but that shouldn’t be happening.  What do we do next?  The first thing I checked is the data that we’re actually pulling.  I have the ability to pull back “all” in the requested_attributes.  Starting there, if I decrease the scope of data I’m requesting and limit it to something like “primaryemailaddress”, all of a sudden everything is happy.  Strange…  Pulling “all” again… nope… null…

The API is returning a ton of data, but we typically only care about the result data and not all the other messages, logs, and other general fluff that comes back from the API… until now.  In my function, you’ll see that we have a line with

where we’re getting out results back from the API. In this case, I want to see all the messages back from the API to see if we’re receiving any errors or other useful information.  To do that, we’re going to change that line to

so we get back everything. That will give us the full response from the REST API so we can see what we’re getting back and play around with all the data in the response.

After re-loading the function, I’m going to re-run my command and see what I get back. (Sparing you the ugly output, there was nothing useful is the messages that were returned in the huge glob of JSON I got back.) So, let’s dump it everything in a variable so we can play with it a bit:

Next, let’s try piping that through ConvertFrom-Json and then we can parse through the data to see what’s going on:

Yeah buddy!  An error!  Now this is something we can actually work with!  “Cannot convert the JSON string because a dictionary that was converted from the string contains the duplicated keys ‘persondirectoryid’ and ‘PersonDirectoryId’.”  BINGO!  It seems our REST API JSON feed is giving us duplicate data in the form of one attribute in lower case and another in camel case.

As a short term fix, we can use “.Replace” to replace the bad data that we’re getting so things work properly:

In this case, we notified the team that owns the API application and they were able to correct the issue with the duplicate attribute.  Though, this does bring up an interesting shortcoming of Invoke-RestMethod:  in this case, it had no tolerance whatsoever for the invalid data.  Both entries were the same except that one was all lower case and one was camel case.  I guess in a perfect world, it would be nice if there was a -CaseSensitive parameter to allow different cases of entries or some other way I could -IgnoreErrors or –DropErrors.  But, bad data is bad data and fixing the data fixed my problem in this case.

Fix XRE-10000 Error During Comcast/Xfinity X1 Activation

Comcast/Xfinity (or whatever they want to be called now) emailed me a month+ back and offered me the opportunity to upgrade our ancient 5-6 year old receivers to their new X1 Entertainment Operating System receivers.  Unfortunately, after plugging them in, we got hit with the error, “Give us a call.  Please call 866-512-2218 and we’ll get this taken care of for you.  Device Serial Number: xxxxxxxxxxxx  XRE-10000”  Three hours on the phone later, here’s how I was able to get it fixed.

Error XRE-10000 on a Comcast Xfinity X1 Receiver
Error XRE-10000 on a Comcast Xfinity X1 Receiver

Call Comcast

Here’s the part that isn’t fun… you HAVE to call the number on the screen.  I was able to get a live human being pretty quickly when I called around 10:00 AM on a Saturday in December.  If you’re an existing Comcast/Xfinity customer that is upgrading to X1 receivers, there is a rate code that needs to be adjusted on your account.  When you get a human being, tell them that you want them to check your “rate code.”  This seems to be a really common problem, but not all the operators know about it.  When I called, the first operator (Jamaican call center) was able to correct the issue on her end for my living room receiver, but transferred me to a U.S.-based call center for my bedroom receiver.  The U.S.-based operator was unaware of the issue and dispatched a tech even after I explained the issue to her.  If this happens to you, take the appointment with a tech but call back and get someone else.  A tech in your home is most likely going to call their internal number and do the same thing you could have done.  If you call them back you may be able to avoid having to wait for a tech to come out to your home.

Check the DVR

Make sure to check that your DVR is working if you’ve ordered it.  Our DVR was not working and I had to call back… again.  There is yet another activation that has to be performed on your account for your DVR.  The first operator may do it; or, you may have to call back again like I did.  Even after the activation of DVR on our account, it still didn’t work and the 3rd operator (Philippines call center) had to dispatch a tech.

Signal strength

The tech arrived at our house and determined that the signal we were getting in the house was low due to a bad splitter outside the house and that was causing some issues getting to the X1 service as well as the cloud DVR.  (As a side note, if anyone at Comcast does read this, the tech that came to our house was absolutely awesome!)  The X1 receivers apparently use the same frequencies as your cable modem and can be very sensitive to interference and bad infrastructure.  If the line from your house to the street isn’t the newer orange quad or tri shielded cable, it may be time to get them to run a new line to your house.   In our case, the receiver was so new that our tech couldn’t identify whether or not it was capable of DVR functionality.  He replaced our receiver and we tested everything before he left.  If you do have a tech come to your home, make sure that everything is working to your satisfaction before they leave or that you’ve got the work order information on any additional repairs they need to perform like replacing the amp outside your home or replacing the line to the street.

Get the most from Microsoft Ignite (When you’re not attending!)

igniteOver the next few weeks, the flow of information about Microsoft Ignite is going to move from the slow trickle we’ve been seeing to full on flood as we get closer and closer to May 4th. The Session Catalog was made available today and, from what I’ve read so far, it sounds like more sessions will be added leading up to the conference. Now, I’m just waiting to see Joey Snow and Rick Claus return with everyone’s favorite video series, The Countdown Show, for TechEd Ignite. We’ll also seeing more planning information to help you make the most of your time while attending Ignite. But, what if you aren’t going to be able to make it to Chicago in May?

I guess I should admit it publicly… I won’t be making it to Ignite this year. 🙁 We’re in the process of expanding our family through adoption and have a lot going on right now. If you have been in that process or know anyone that has, you know that it is a monumental process (the paperwork is staggering) and, depending on circumstances, could mean that we have to travel at a moment’s notice. Needless to say, being in Chicago, IL for a week while my family is in Atlanta, GA just wasn’t in the cards this year with everything we have going on.

If you’re in the same boat as me and can’t attend Microsoft Ignite in person, there’s still quite a bit you can do remotely through the miracle of modern technology:

Follow the action Twitter. Seriously, get on Twitter. If you attend Ignite in person, you’ll hear the same theme… a lot. Most of the speakers are on Twitter and quite a few will take questions during and after their sessions via Twitter. It’s also a great way to start finding and connecting with other IT Pros with common interests for those introverts like me.

Follow the hashtags. Even if you’re not attending Ignite in person, different Twitter hashtags can be a treasure trove of information. The official hashtag for the event is #msignite; but, you can also follow some of the common product hashtags depending on your interest: #Azure, #Office365, #WinServ, #Windows, #SysCtr, #mscloud, and #sqlserver, just to name a few. (I’d probably never hear the end of it if I didn’t mention #TheKrewe hashtag either. Though, it’s probably going to make you wish you were in Chicago instead of following along online.) There are also hashtags for specific sessions, but I haven’t seen the format that is going to be used this year; as soon as I see it, I’ll update this post.

Watch the keynote. The keynote for large Microsoft conferences like Build and TechEd have always been streamed live on Channel 9. Microsoft Ignite’s keynote on May 4, 2015 will feature CEO Satya Nadella along with other senior executives from Microsoft. With the impending release of Windows 10 and updates to other Microsoft products, it’s safe to assume there will be big announcements during the keynote that you won’t want to miss.

Watch the live feed. Based on past history with TechEd, Channel 9 will most likely also have a live feed of selected sessions during Ignite. Between sessions, there are interviews and other presentations from the Channel 9 area of the Expo Hall. (It’s also very likely there will be a live Patch and Switch recording during the week also.)

Watch recorded sessions. Make sure you check out the Session Catalog and take note of sessions that look interesting or relate to your job responsibilities. Video from the sessions along with PowerPoint slides will be posted on Channel 9. I personally like that there are downloadable versions of the video I can put on my tablet.

Find a local IT Pro user group. Last but not least, find a local user group. Just because you can’t go to Ignite doesn’t mean that there aren’t plenty of other learning and networking opportunities out there. Many of the speakers at Ignite also speak at local user groups and conferences. If you happen to be in the Atlanta area, check out the Atlanta Windows Infrastructure and Virtualization User Group (WINVUG), the Atlanta Systems Management User Group (ATL SMUG), and Atlanta TechStravaganza.  I’d love to see you there!

How Symbolic Links Make Server Migrations Much Easier

This is a guest post from Joseph Moody at DeployHappiness.com. More information on Joseph is included in his author box following this post. Thanks, Joseph!!!

 

Symbolic Links are the most powerful file server tool that you aren’t using. Occasionally called symlinks, these advanced shortcuts allow you to perform some serious smoke and mirrors when accessing data.

Consider the following common scenarios:

  • An application checks for data in a certain location. You would rather store that data elsewhere.
  • Old software needs to write to C:\. You want it to write to %ProgramFiles%.
  • You wish to move data to a new share but don’t want to break existing shortcuts
  • You need an easier way to migrate paths from standard shares to DFS Namespaces

In all of these cases, symbolic links provide a solution.

 

Creating Your First Symbolic Link

If you have ever installed Windows Vista or higher, you’ve created a symbolic link. That hidden Documents and Settings shortcut in the root of C: is technically a symbolic link. It refers to C:\Users for compatibility purposes.

Windows symbolic link Documents and Settings linked to Users

Making a symbolic link is easy. To create one, you will use the mklink command. Fire up an administrative command prompt and type mklink /? to see the syntax.

Command Prompt showing mklink syntax

As an example, let’s create a symbolic link that redirects a folder from the root of C: to %ProgramFiles%. First, create a folder in C:\ named data. Populate this folder with a file or two. This is your source folder – the folder that you will be moving.

Cut this folder from C:\ and move it to %ProgramFiles%. In your administrative command prompt, type: mklink /D C:\Data “%ProgramFiles%\Data”

example of making a symbolic link with mklink

You should now see a shortcut in C: named Data. Its type though should read file folder. When sorted by name, it should also appear as a folder (one advantage over shortcuts).

If you open the Data folder, you should see the exact content that you moved over to %ProgramFiles%. As a test, open a second Explorer window and navigate to %ProgramFiles%\Data. Create a new text document – it should appear in C:\Data. Like Magic!

 

How Will You Use Symbolic Links?

The mklink command supports way more than we just showed. At times, you may have to use a directory junction, create hard links, or specify relative target paths.

Working with symbolic links is the fastest way to master these advance parameters. If you want to learn more about symbolic links, check out these three links:

Stop Mouse and Keyboard Theft with a Cable Lock and Washer

I recently had to deal with the disappearance of several keyboards and mice from computers that are set up in a semi-public hotelling area.  I received a support request from someone that noticed that some of the computers were missing either a keyboard, a mouse, or both.  We had no reason to believe they were stolen and were most likely taken by a well-meaning employee assisting a co-worker or fixing their own issue.  We keep a stockpile of extra keyboards and mice; so, replacing the missing keyboards and mice was trivial.  However, we still have to account for the inventory and really need people to contact us when their equipment breaks.

The solution?  A cable lock and a washer that cost less than $0.25.

inexpensive washer

The cable for the mouse or keyboard is looped through the washer.

mouse cord looped through the washer

If you find a washer with a large enough hole, you can loop both the keyboard and mouse through.  If the hole isn’t large enough, you may need to increase your budge by ~$0.25 for each PC.  🙂

keyboard and mouse cord looped through the washer

As you can see in this up close shot, the end of the USB cables can’t be pulled through the washer.

keyboard and mouse cord looped through washer up close

Many of our computers are already attached to desks as a theft deterrent using a cable lock. All we had to do was disconnect the lock from the back of the computer and pull it through the loop created on the cables.

security lock pulled through cable loop in keyboard and mouse

Obviously this isn’t completely foolproof, but should be enough of a deterrent to keep the casual keyboard/mouse thief from walking away with your equipment.

Prevent the “Your browser has been upgraded” tab in Internet Explorer

Microsoft’s June Cumulative Security Update for Internet Explorer (MS14-035 / KB2957689) had a change that caught many IT departments off guard. If you’re in an environment running Windows 7 with either Internet Explorer 9 or Internet Explorer 10 your users may have received an additional tab that opened after the reboot from their monthly updates applying:

Internet-Explorer-Your-browser-has-been-upgraded
Your browser has been upgraded… sort of… with a monthly security patch…

Initially, users were redirected to http://windows.microsoft.com/en-us/internet-explorer/products/ie-9/windows-internet-explorer-9-privacy-statement, a rather lengthy privacy statement for Internet Explorer 9. After customer complaints user feedback, Microsoft altered the redirect to send users to http://windows.microsoft.com/en-us/internet-explorer/ie-9-welcome-upgrade3.

Unfortunately, this still isn’t expected behavior in a corporate environment. End users tend to either ignore something like this completely or open a help desk ticket costing the IT organization money in the form of the help desk request. The problem is compounded by: (#1) Microsoft not warning corporate IT departments this change was coming, (#2) Microsoft not giving corporate IT departments a way to suppress the extra tab with the warning, and (#3) some users receiving the additional tab every time they open an IE window instead of seeing it just once.

The good news is that this extra tab can be suppressed with a Registry entry. The easiest way to do this in a managed environment is with Group Policy.  In a Group Policy Object (GPO) that applies to user accounts, go to User Configuration > Preferences > Windows Settings > Registry.  Right-click on Registry and choose New > Registry Item.

Group Policy Management Editor Add new Registry entryIn the Properties for the new Registry item, set the following:

Action:  Update
Hive:  HKEY_CURRENT_USER
Key Path: Software\Microsoft\Internet Explorer\Main
Value Name: PrivacyPolicyShown
Value Type: REG_DWORD
Value Data: 00000001

PrivacyPolicyShown PropertiesObviously this won’t help you for the hordes of end users that have already received the extra tab, but it should prevent anyone logging into a system for the first time from seeing it down the road.

Encourage Users to Submit a Ticket Instead of Emailing You Directly With a MailTip

Exchange-2013-LogoHow many times has this happened to you? You go on vacation, to a conference, you’re inundated with email, or for any of a hundred other reasons you don’t see a support request from an end user come in. Fast forward a few days or weeks and the end user is concerned that their issue hasn’t been resolved. [And we all know that “concerned” could be anything from genuine concern for your well being (“You always respond so quickly!”) to concern that your job performance should be discussed at the highest levels of your organization for not responding to them within 5 minutes.] So what’s the problem? The end user emailed you directly instead of submitting a support request through a ticketing system… a ticketing system that, most times, alerts a team of people about the problem so that their issue can be handled when you’re out of pocket.

We all know what happens… end users find a favorite “computer guy” or you’re a one man shop; but, support requests start coming directly to you that should go through the ticket system. Short of outright refusing direct support requests, it can be difficult to get some people to submit tickets.

Use an Exchange MailTip!

One creative way I’ve seen companies handle this is by setting an Exchange MailTip for certain IT Pros.  Here’s how to do it in Office 365:

Go to the Exchange Admin Center at https://outlook.office365.com/ecp and click on Mailboxes.

01-mailitp_for_it_supportHighlight your account (or any other IT Pro) and click the Edit button.

02-mailitp_for_it_supportClick on MailTips and enter the message you want to be displayed.  When you’re done, click the “save” button.

03-mailitp_for_it_support There’s a slight lag from when you set a MailTip and when it shows up for end users. When the MailTip starts showing up, end users should get your warning that they should submit a ticket instead of contacting someone directly.

04-mailitp_for_it_support