Hello and welcome to another
wonderful episode of the VCD Roundtable.
This is episode 48, and I atleast have full control of
my computer, microphone, and camera.
Unlike some other people (in the chat), I
just actually solved it on my own.
So with this episode, we are going to look into
Orchestration and Automation,
which is at least when it comes to Cloud
Director, far more a topic for Matthias.
So before we get to Matthias, Toby, any opening words?
No, absolutely not from my side.
Hello and welcome to episode 48.
Yeah, let's hand it over to Matthias
and listen what he is referring to.
Hi guys, this is Matthias.
Yeah, I'm not in control of my mic anymore, because my
MacBook Pro refuses to accept the microphone I configured.
It insists of using my iPhone for whatever reason.
But yeah, nothing I can automate.
Cloud Director Orchestrator and Automation.
Automation has multiple approaches.
For example, we use automation to onboard new customers.
Onboard new customers, for example, is not like a...
We're not automating it to have it
more or to have it onboarded faster.
So from my point of view,
Automation is not always to speed things up.
On the other hand, a big part of
Automation is to have configuration consistency
and there's standardized implementation of something.
We all are human beings, so we are pretty
aware that you just fat finger something.
Maybe you select the wrong allocation model for the OVDC
and then you configure everything.
And then it's like, "Oh, I selected the wrong one."
So delete everything.
So that's a big part of Automation as
well, to have standardized configuration.
I don't know how many guys of you have already played with
Automation on Cloud Director.
So we have two main pillars to automate Cloud Director.
First, we can just use the APIs and build a workflow to do
certain steps, one after the other,
to achieve a goal.
On the other hand, we can use
Cloud Director Event Broker Services
to trigger based on certain events and external systems to
kick off an automation process.
So these are the two main pillars we have.
From my point of view, and I have a big grin on my face
because Yves and Toby both told me,
"Orchestrator is dead" years ago. It's still there.
And it will be there in the future and forever.
So I'm a big "Orchestrator" fan.
On the other hand, I have to admit there is a great PowerCLI
implementation or a cmdlet
for the Cloud Director API as well.
So everybody, feel free to use PowerCLI as well.
I'm more the "Orchestrator" guy,
but that's just my personal opinion.
Most important thing is to automate stuff.
So PowerCLI and Orchestrator can both
be used to access the Cloud Director API
and automate things.
I still stick with the example: tenant onboarding.
Why am I using this example?
Because tenant onboarding includes many, many steps.
So first, guys, what do we need?
We need to select the PVDC.
We need to create an OVDC, select an
allocation model, choose resources.
We deploy an Edge gateway.
We need an OVDC network.
We need IP space allocation, maybe some
default firewall rules in the Edge gateway.
Some users create things like this.
So a ton of stuff which needs
to be done to onboard a new tenant.
That can be done in an automated fashion.
So all these steps I've mentioned can
either be built in PowerCLI or Orchestrator.
Before we dig more into that topic, I need
to make you guys aware that Cloud Director
currently offers two APIs.
We still have the old standard REST-based API.
We still have the old standard REST API.
And the new one is called the Cloud API.
I have to admit, the Cloud API
is the new stuff, its cooler.
I'm refusing to say the better API.
It's an easier API compared to the standard one.
But the Cloud API still misses a lot of important stuff.
Everything about automating and managing virtual machines
and vApps is still not in the Cloud API.
And I don't know when it will be part of the Cloud API.
So what we need to think, if we cover automation, we have
the two main aspects I think we're
going to cover is PowerCLI and Orchestrator.
Power CLI is shipped with its cmdlets for Cloud
Director, and they can be used if
you need to implement something and you don't have a
cmdlet, use PowerShell to implement
the REST API call.
And on the Orchestrator side, it's the exact same thing.
So you could implement the plugin
in Orchestrator and use the plugin.
Honestly, it is nice, though.
I have to admit, it's helpful.
It's nice.
You can solve a few very basic
tasks really fast with the plugin.
It has an implicit authentication,
which you can use in JavaScript code.
So that's pretty nice and straightforward.
But I honestly tend to implement
all the Cloud Director calls via REST.
The plugin has not all the methods and API calls
implemented, so you are dependent on
the development team of the plugin if a
certain feature is available via the plugin.
If not, you need to build it in REST anyway.
So that's the reason I prefer
the REST approach in Orchestrator.
And what I start with, I build a template.
So a few templates that I can use over and over and over,
because I don't want to build
and structure every single workflow to rebuild every single
REST API call over and over and
over.
So it's easy to just grab a template and change a few lines
of code, the method, the URL,
or some inputs, outputs needed, and then use the same
workflow over and over and over.
So it's also a very standardized approach
how to build workflows for automating, in
our case, Cloud Director.
But I use the same mechanism if
I automate NSX or other stuff.
With Cloud Director, as long as we use the Cloud API, we
have a great Swagger documentation
of the API built into the Cloud Director product.
On the top right, just click the
question-- I think it's the question mark.
And then you have a menu entry saying API Explorer.
And then just navigate over there.
Please be aware, Cloud Director
is still a multi-tenant product.
So you have an API documentation
for the tenants and for the provider.
And they are different.
Always keep that in mind.
Oh, what just pops up in my mind is:
There is a very cool mechanism, guys.
If you as a CSP would like to offer your tenants API access
to Cloud Director, there is a new
session to generate user tokens.
So it's somewhere in the UI.
On the top right, where the username is,
you can go to User Preferences, I think.
And there is a section or a menu
entry saying "generate access tokens."
So you can use that access-based
mechanism for the API for your tenants as well.
It's pretty nice.
Also for service accounts and other stuff.
So it gets even more secure.
Nevertheless, so the Cloud API documentation is great.
It's easy to use.
It's built into the product.
You have all the URLs and like a Swagger
documentation is, you have all the tryout,
functionality, copy, paste, etc., all that kind of stuff.
If you have or if you need a REST API call, which is not
part of the Cloud API, you need
to move over to the standard API.
Yes, there is a documentation.
I would phrase it, it's not as easy to use
as the Swagger documentation of the Cloud API.
It's a website in VMware code
and best of luck to find something.
Honestly speaking, I prefer if I need to deal with the
standard API and I have no clue how
a certain API call works.
I prefer using the developer tools of my
favorite browser and just grab the call, the
UI fires in the background against the API because the
Cloud Director UI does or is just
consuming the REST API in the background.
So there is no secret behind that.
Use developer tools, grab the REST API call from the
backend, which is monitored by the
developer tools and then you
have the URL, maybe some data model.
So you can go to the header section, validate which header
it is, which API version it is
and then check the payload.
If it's a put or a post, grab the data model.
Sometimes you will find some hidden options in the data
model if you're using the developer
tools.
So even with the Cloud API, not every
single parameter of the data model is specified.
And then you just use this stuff, you
have sniffed and used a different data model.
It also helps which parameters are mandatory, which are
optional because I, for example,
sometimes struggle with, "oh, I need parameter ABC or where
do I get this information from?"
Maybe it might not be available.
So it's easier to be like, "oh, it's just optional.
I just leave it empty or remove
it from the data model as well."
And then you can do it by REST.
I think that's a pretty straightforward approach for
automating Cloud Director and also basic
tasks.
And then you just move on for NSX and other products.
If you have Cloud Director,
please automate NSX via Cloud Director.
I think that's something very,
it's always like, oh, yeah, thank you.
I saw it in your face, right?
So it is important to say, even if you
automate stuff, stick with the rules.
Cloud Director is the leading system.
If you've Cloud Director and your tenants
are consuming NSX, automate on an NSX via
Cloud Director.
Except, unless you're automating something in NSX, which is
meant to be a managed service
for your customers.
So it would be configured in NSX manually anyway.
One of the features which just comes up
is, for example, Distributed IDS/IPS.
That's a managed service from a CSP perspective.
Okie doke.
So that's basic automation.
The even more interesting part is,
yes, Yves, you want to share some thoughts.
You need to unmute yourself.
Yes, I think the big question for me is, on one end you
said, you don't know when the
Cloud API is going to be finished.
I think the answer to that is very, very straightforward.
It's never going to be finished, because that requires
additional development investments into VCD.
And VMware was very clear that besides patches or anything
else, they are not going to invest
a penny into VCD anymore.
I would not, to be honest, expect anything really coming
back into that one, especially
looking at how long it took them to
get the API to where it currently is.
And I would not expect much to the other ones.
They promised that we will get similar API in the new
systems, but they did not say similar to what.
So whether that is the old-old one, the Cloud API or the
current Automation API, which they
also sometimes refer to.
So the question is, if someone didn't start so far on their
automation journey, what would
be your advice at the moment to
automate anything around Cloud Director?
Would you still advise them to go directly REST?
Do you advise them to use any of the existing packages?
Or maybe use something more like
an infrastructure as code approach?
Yeah.
So I still think the plugin is great, but,
and there's a big but, if you start using
the REST approach, you're more flexible because all the
other products support the REST API
as well.
NSX, the vCenter server, SDDC Manager, even Operations
manager, Operations for Logs,
or it's a bit more hidden, but they have a REST API and
they all use the same mechanism.
If you are familiar dealing with one REST API, you're
familiar dealing with all the
other REST APIs as well.
The only big difference between all
the APIs is authentication, right?
So you once need to figure out how to build
Cloud Director authentication via REST, or
you need to figure out how to build the authentication for
NSX because they're different.
But at least speaking for VMware by Broadcom, the
documentation, how to authenticate against
the REST API is properly written for
every single product I've dealt with so far.
Toby?
Just to add here one thing, especially for those who really
begin to start now with Automation
or playing around Automation, let's call it that way,
because if you really have already
a use case that you would like to automate something, then
maybe it is not the best idea.
But if you're really starting now from scratch and say,
hey, I would like to have people
inside and look and play around with it, I would heavily
recommend to play around with
the REST API of the Operations Manager, especially for the
future, more and more becomes the
centralized API for the whole VCF game.
So if you now really start with the REST API, then I would
highly recommend that you play
around with the API of the Operations Manager.
That's a good mentioning.
Thank you.
So if you're familiar with REST,
you're good to go with all the products.
And it's not related to only VMware by Broadcom.
But I have to admit, I don't know anything about the
documentation quality of REST APIs
of other vendors.
So I'm not commenting.
So that's very useful.
You can even add Orchestrator into the service library of
Cloud Director, and then you can
use a VCD UI to kick off or to start VRO... it's Aria
Automation Orchestrator -- a VRO workflow.
And if you build the input form properly in VRO, it's
identically displayed in Cloud Director.
And if you're really brave, you play a bit with the actions
in Orchestrator to populate drop-down
menus and stuff, and then you can add filters based on
organization ID, which might make sense to have
an input form of a workflow, a multi-tenant based on Cloud
Director, because Cloud Director knows the
organization ID, OVDC user ID, and stuff like this.
And the combination with roles and rights enables the
visibility of certain objects like an OVDC network.
So that's this part.
Let's move to the dark side of
Automation -- event-based stuff.
I think that's even a more challenging area to automate
infrastructure behavior, because
that's exactly what we're talking about.
So within Cloud Director, if you
go--so that's the provider context.
Within the provider context in the administration section,
there is a menu entry called Extensibility.
That's where you add the message broker to the system.
And on the second tab, you can enable events.
And then you can select--you can either trigger that Cloud
Director should send all events to the event bus,
or messaging bus, or to certain events, and you can choose
if it's blocking or non-blocking.
So non-blocking means Cloud Director sends
an event and continues with its workflow,
or if it's blocking, Cloud Director fires the event,
triggers the event, and waits for the response.
So the response could be successful, failed, or cancelled.
And it sits there until it reaches the default time out.
And that's five days.
So you can wait for an external system.
That might make sense if you ramp up a new virtual machine,
and you need to grab an IP address from an external IPAM.
It might make sense to wait for it, because you need to
inject it into the boot process of the guest... for example
And of course, if you use the event-based automation, you
need to implement a RabbitMQ message bus in the backend.
There are some hacks to access the MQTT,
so the internal Cloud Director message bus,
but I'm not the biggest fan of
it, because I didn't understand it.
So it's a bit more of a hack.
So I don't preferr that option, because I leave the
appliances as they are, because from a support perspective.
So I'm not changing anything in the appliance.
Just attach the message bus system, and then you attach
Orchestrator, on the other hand, to the message bus.
Do some configurations to route messages.
So I have a nice blog article out there.
It's like zillions of years old, but still accurate.
And then you just consume the messages, and based on the
message, you start the workflow.
Hint, or today I call it a pro tip.
If you deal with messages, there is something out there
called the VMware Cloud Director Notification Package.
If I have it in my head correctly, I think it was built by
Christophe Decanini initially.
I think, so he's a great guy.
So that notification package enables something very
specific, which is, you can
collect a message from the message bus.
That's Orchestrator out of the box behavior.
But you have a message runner workflow, and the message
runner workflow converts the message
body into Orchestrator usable objects.
Which is very helpful, because
otherwise you need to start dealing with XML.
And dealing with XML in Orchestrator with the Rhino
JavaScript engine is not even a nightmare.
It's like, I don't know. No, it's not working.
Yep, oh, Yves volunteers.
So from now on Yves is doing all the XML stuff for you guys.
I'm not touching Orchestrator
So the notification package.
I have something to write my code now.
I tried it and it can't deal with it because the Rhino
engine in the JavaScript engine
version in Orchestrator is way too old.
And does not understand the XML header fields and stuff.
Which tells me a lot about the product.
It's a great product. It's old. It works.
But coming back to one sentence before, are we sure that
AMQP still is supported in 10.6?
Because MQTT should now be finalized and working. From what
I have heard and what I have tested.
I don't get the access method
and RabbitMQ is still supported.
Does Hock charge an extra license
for RabbitMQ now that he owns it?
Please not.
Just asking.
I don't know. I didn't read anything.
So another very very common use case for the Cloud Director
Notification package is or are the blocking tasks.
If you're using blocking tasks, the
message runner workflow converts automatically.
Or no, let's phrase it differently. The
message contains the blocking task ID.
And the Cloud Direct Notification package, the message
runner workflow converts that ID
into a vCloud blocking task object.
So that can be triggered via the plugin.
So these are some rare cases which
are just the plugin is so useful.
I once built a whole rest mechanism with blocking task ID
to resume or cancel whatever.
But honestly with the plugin, it's just two lines of code.
And then you just trigger the blocking task. It's
successful, continue, it's failed, it's aborted.
And you can also implement based on that behavior some
error handling in the workflow.
Some rollback or whatever is needed.
So that's the event based
automation approach out of Cloud Director.
And the Notification package also enables you to filter the
messages based on organizations, based on users, based on
event types or whatever is needed.
Is part of that notification package.
The only thing is, the message runner workflow is not
working anymore out of the box.
Second pro tip.
The methods of the date time object have changed.
You just need to go into the code, it's around line 140,
and just change three or four methods.
And then it's all good. You just need to
alter the workflow and then it's working again.
It tells the error message tells
you exactly what's not working.
Just validate that the methods are wrong, use the correct
methods and you're good to go.
That's the second pro tip.
Yeah, so that's...
So maybe one last idea based on a statement of Toby.
Because Toby said, and that's
something you always should keep in mind.
If you're new to the automation game and you want to start
with automating stuff, I have two tips.
First, never ever just sit in front of the computer and
tell yourself, "I'm now going to automate something."
It's not working.
It's not working, because you don't have a goal. You cannot
achieve the goal. You will never have a result.
Always come up with the use case and implement the use case
and please start with very simple ones.
And that's my second point.
Never ever start, if you're new to the game, start
automating virtual machines or vApps in Cloud Director.
That also refers back to the vCenter server. Please do not
start automating virtual machines.
Because these are from the amount of methods and properties
they have, they are by far the most complex objects.
Just start with simple tasks. Let's call it that way.
Yes, exactly. Toby, you're completely right.
If you're struggling in the beginning, what could be an
easy thing to automate just to play around with it?
Create an Edge Gateway in an existing org.
The OVDC itself is a very complex object
as well because of the allocation model.
That's the only reason. And I have
another, I today have a ton of pro clips.
If you would like to build an OVDC with Flex as an
allocation model, the REST API is the only way to go
forward, because the plugin is
not aware of that allocation model.
It's that simple.
It was just introduced about five years ago.
Yeah, so it's like it never went to school.
Good. I think that is a good starting
summary around Automation Orchestration.
If you have any questions in any of these areas, feel free
to ping the master of old-school automation called Matthias.
No, his name is Sascha.
Well, Sascha does the fancy new stuff.
And also, if you have any other topic suggestions, feel
free to drop them somewhere in the show notes, no matter
which platform you are using, so that we get that.
Do we know already the topic for
the next recording in two weeks?
Okay, I see silence.
Okay, I see silence. So that means you should better follow
us on social media and see what we are going to announce,
what is going to be Episode 49.
And then we better have a good
idea of what's going to be Episode 50.
But nevertheless, I would say
thank you all for listening in.
You can find this also on any of the
podcast platforms and subscribe over there.
Any closing words, Toby?
Not from my side. Start with automation
as soon as possible and have a happy day.
Yeah, guys, go into automation. It's just cool. It's better
than just clicking and watching
progress bars moving and stuff.
And then you free up time to learn
new topics and do even cooler stuff.
The first steps are pretty hard to
take, but get used to it, go into it.
It's not that difficult once you get used to the mechanism.
I think, and I will end with a side take, I think it was
William Lam who said, "Before I do
it once, manually, I automate it."
A good man, a wise man.
Thanks for that closing statement. See you all on one of
the next episodes and automate the world.
Thanks for tuning in, guys. Bye.