disc-hanging

Sometimes using database views rather than database tables can be very helpful for querying aggregated data. Typically in the Django ORM, a database table is tied to a Django ORM model, but we found it is also possible to tie a database view with a Django ORM model. By doing so, you can hide the data aggregation logic on the database level (in view creating SQL). And most of the ORM features, like double underscore foreign key lookup, still work for the model tied to a database view. In this blogpost, I’ll walk you through a simple example to illustrate how.

The regular models

Our example uses the following regular models. Each model has a database table tied to it.

The Charge model is used to store the charges. It has a foreignkey to the user model and an optional foreignkey to the job model. Not every Charge record can be associated to a job.

The database view

The database view is created by following the SQL and it can be injected into a customized data migration with the raw SQL execution command.

This view sums up all the hardware charges and software charges for the Charge records with the same job, user, and month. It contains the following columns:

The next step is to create a Django model which maps to this view so we can use Django ORM to retrieve the data from the view.

The model for the view

We created the model below to map the database view.

Although the model above looks very similar to a regular Django model, there are three differences worth noticing.

  1. For the foreignkey fields, we need to specify an extra kwarg: on_delete=models.DO_NOTHING. Although normally we won’t try to call delete() on the ChargeSummary model because of the view at behind is not writable, without specifying this kwarg would lead to some test cases failure. The might be related to the auto cleaning mechanism for the test database after test cases running.
  2. managed = False needs to be specified in class Meta so that the south or Django database auto migration will ignore that model and won’t try to create a database schema migration for it.
  3. db_table = ‘name_of_the_view’needs to be specified in class Meta to let the Django ORM know which database entity to retrieve the data from.

Query with the new model

With the new model being set up, you can use the regular ORM query methods with no difference compared with the regular Django model.

Get all ChargeSummary objects:

Get all ChargeSummary objects for May 2015:

Get all ChargeSummary objects with a certain job name:

Get all ChargeSummary objects with a specific user email address (suppose you have a 1-to-1 userprofile model associate with the user model):

Of course certain methods like save() or delete() will presumably fail since DML SQLs are not applicable to database views by nature.

Last but not least

If you are upgrading your app to Django 1.7 or later from an earlier version, you need to run

to generate 0001_initial.py and then run the command below to upgrade your database.

Unfortunately, this would fail if you have the database view and the unmanaged ChargeSummary model. The reason is when Django runs the migrations with the fake initial flag, it verifies that the tables do exist in the database for all the models in CreateModel operations in the 0001_inital.py migration. CreateModel operations are created for every model, even those with managed=False. So for ChargeSummary model, it will fail since it can’t find the actual table. Fortunately, we can work around this issue by adding the view create sql into 0001_initial.py, and then run the migrations with the fake initial flag.

Happy modeling and coding with Django ORM.

This article was written by Irwen Song.

Microsft azure logo header

Rescale recently rolled out SAML Single Sign-On login support for our ScaleX Enterprise users. This post will discuss how to set up Rescale as a SAML Service Provider, using Azure Active Directory as the Identity Provider.

Prerequisites to follow the tutorial below are that you have ScaleX Enterprise and Microsoft Azure accounts.

SAML Background

SAML is an authentication protocol used for web single sign-on (SSO). There are 2 entities involved in a SAML deployment:
Identity Provider (IdP) The entity that manages your users’ credentials. Its role is to authenticate users with passwords or other types of keys.
Service Provider (SP) The entity that provides end user applications. It relies on the Identity Provider to authenticate users.

This Wikipedia page has far more detail, but the SAML protocol roughly consists of the following steps:

  1. User tries to access a resource from the Service Provider and the provider needs to authenticate that user
  2. Service Provider redirects the user to the IdP’s SSO endpoint with a user identifier
  3. IdP redirects the user to its own login page if the user is not currently authenticated with the IdP
  4. IdP responds with authentication and redirects user back to the SP’s Assertion Consumer Service (ACS)
  5. SP ACS verifies the authentication came from the IdP and logs the user in on the SP
  6. SP redirects the user to the originally requested resource if they are authorized to access it

Note that while the Identity Provider manages the authentication of the user, the Service Provider still imposes its own authorization rules on that user. Also note that the Identity Provider and Service Provider never exchange the users’ secret key information but instead the Service Provider is just allowed to ask the Identity Provider about the identity of the user currently logged into the web client.

In this tutorial, the Identity Provider will be an Azure Active Directory and the Service Provider will be Rescale.

The configuration we will outline has 3 high-level chunks:

  1. Set up a new Azure Active Directory as a test IdP
  2. Add your Rescale as an authorized SP to access your new IdP
  3. Set up your ScaleX Enterprise account to authorize your new IdP

Creating a test Azure Active Directory SAML Identity Provider

For completeness, we will start by setting up a test of Azure Active Directory. If you already have an AD set up for your organization, you can probably skip ahead to the “Authorizing Rescale as an SP” section.

Start by logging into the Azure management. Go to the Active Directory section and create a new directory with the +NEW button in the bottom left corner.

azure-select-ad

Select DIRECTORY and CUSTOM CREATE

azure-select-directory

Fill in the fields to create your new directory. The NAME and DOMAIN NAME can be the same but should be unique across directories in your organization.

azure-custom-add

At this point, you have just created a new Active Directory. You can now add users to your directory. Select the directory you just created and choose the USERS tab at the top.

azure-new-config

Your Azure user should already in the user list for this directory. Add another user by selecting ADD USER at the bottom.

azure-add-user

Select “User with an existing Microsoft account” and add an email of the user from your domain you wish to add. Fill out additional profile information on the next page and then save the new user.

azure-add-user-name

At this point, we have an Active Directory with one or more users. The next step is to tell our AD to allow Rescale to query for users to authenticate.

Authorizing Rescale as a Service Provider

The Identity Provider must now be told to allow Rescale to query it for logged-in user information.
Select the APPLICATIONS tab at the top and then select ADD at the bottom. Pick “Add an application my organization is developing.”

azure-new-app

Name your application to whatever you like and set it to be a “WEB APPLICATION AND/OR WEB API”.

azure-app-name

On the next page, you start to configure the important bits of your Service Provider application. You should set your SIGN-ON URL to “https://platform.rescale.com/saml2/company ID/sso/” and your APP ID URI to “https://platform.rescale.com/saml2/company ID/”. Your company ID is generally just the name of your company but you can contact support to verify this. In this example, we are adding users using the company code “rescale”.

azure-app-properties

Next we need to set a few additional properties and get the IdP endpoints the Rescale Service Provider will use. Go to CONFIGURE tab and scroll down to the REPLY URL. Set the URL to “https://platform.rescale.com/saml2/company ID/acs/” and then SAVE the change.

azure-app-configure-reply

Next, let’s note the endpoints we will need to configure our Rescale account to use this IdP. You should copy these endpoints:

  • SAML-P SIGN-ON ENDPOINT
  • SAML-P SIGN-OUT ENDPOINT
  • FEDERATION METADATA DOCUMENT

Next, we need to retrieve the Azure entity ID it uses to identify itself with the Service Provider and the X509 certificate it uses to sign SAML responses.
Open up another tab in your browser and go to the FEDERATION METADATA DOCUMENT URL you just copied. Copy both the “entityID” attribute and the first “X509Certificate” element in the document.

azure-metadata-get-fields

At this point, we have everything we need from the Azure console. It is time to move over to the Rescale platform to configure your Service Provider.

Authorize your Identity Provider in ScaleX Enterprise

Open up a new tab and log into https://platform.rescale.com as a company administrator. Select Company Administration in the top right user drop-down menu, then select the Settings tab and scroll down to the “SSO (Single Sign-On)” section.

rescale-sso-settings

You can now fill in the relevant fields here:

Service Provider saml:NameID Format attribute: For Azure, this should be set to “urn:oasis:names:tc:SAML:2.0:nameid-format:persistent”
Name of Identity Provider email field in ACS response: For Azure, should be set to “http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress”
Identity Provider EntityDescriptor Entity ID: This should be set to the “entityID” value copied from the Azure metadata.
Identity Provider SingleSignOnService URL: This should be set to “SAML-P SIGN-ON ENDPOINT” from the Azure endpoint list.
Identity Provider public X509 certificate: This should be filled with the X509 certificate copied from the Azure metadata. You should ensure there are no line breaks in the middle of the certificate contents.

Note that the first two fields are the same regardless of your particular AD deployment but the last 3 are different and are based the values you captured above. These three values will differ from what is shown in the screenshot.

Check the following checkboxes:

  • Active
  • Encrypt NameID
  • Sign AuthnRequests

For your initial test, you should also select “Create any user who can authenticate with SSO”. More on this in the next section.

Then click Update SSO Settings. With that, you should be able to log your user in using your AD IdP! To test it, log out of Rescale via the upper right user drop down. Go to the Rescale SSO login page and try to log in as one of the users you authorized by your Active Directory provider.

SP vs. IdP initiated login and invites

Users of your ScaleX Enterprise account can log in with SSO through either of these endpoints:

IdP initiated logins https://platform.rescale.com/saml2/company ID/sso/
SP initiated logins https://platform.rescale.com/login/sso/

The former is the more “direct” link. It redirects straight to your IdP SSO URL and does not require the user to enter any credentials on Rescale’s side. The second page takes a user email and then tries to route the user to the correct Identity Provider. This page is meant for users that start by going directly to https://platform.rescale.com but still want to log in with SSO.

As mentioned in the previous section, you can choose to either allow any user authenticated by your IdP to log into Rescale under your organization, or you can only allow users to create Rescale accounts who have been explicitly invited. In order to enforce the invite-only restriction, you should go back to the SSO settings section in the Rescale Company Administration panel and select “Only create invited users” and save those settings. Now to invite users, go to the Members tab at the top, click “Invite Members”.

rescale-invites

You can then enter a list of emails of the IdP authenticated users you want to enable access for on Rescale. These users will then be sent invitation emails with the IdP initiated login mentioned above.

At this point, you should have a secure Active Directory configured to allow Single Sign-On on Rescale. You can now manage your users access to Rescale by either controlling user authorization in the Rescale Company Administration portal or controlling user authentication to Rescale via your Active Directory.

This article was written by Mark Whitney.

visualization

San Francisco, CA – July 7th, 2015 – Rescale announces a 3-part webinar series that helps CIOs and IT professionals make a smooth transition to cloud HPC for engineering and science simulations. The webinar series will be held on three consecutive Wednesdays- July 29th, August 5th, and August 12th of 2015, at 8:00 am Pacific Daylight Time (11:00 am Eastern Daylight Time).

Today, a responsive IT environment is critical to support the dramatically increasing and highly variable user demand for simulation. Enterprises are looking for solutions to help them effectively transform their legacy on-premise IT infrastructure into a dynamic environment that is high performing, scalable, and secure. The cloud enables these organizations to further improve the business bottom line with better products and an accelerated time to market.

This 3-part webinar series will cover three distinct, yet interconnected, topics including, an enterprise roadmap to elastic computing, integrating on-premise HPC with cloud HPC, and an enterprise cloud with full IT control and security. Rescale’s ScaleX Enterprise platform will be discussed as a platform to help bring cloud and on-premise resources together into a consolidated environment that provides elasticity and responsiveness to drive innovation for next generation product development.

Rescale invites CIOs and enterprise IT professionals, simulation engineers, and engineering managers to join this webinar series. Registration links for the 3-part webinar series are provided below:

Part 1: An Enterprise Roadmap to Elastic Computing                 Register Here
Date:     Wednesday, July 29th, 2015
Time: 8:00 AM PDT/11:00 AM EDT

Part 2: Integrating On-Premise HPC with Cloud HPC                  Register Here
Date:     Wednesday, August 5, 2015
Time: 8:00 AM PDT/11:00 AM EDT

Part 3: An Enterprise Cloud with Full IT Control and Security    Register Here
Date:     Wednesday, August 12, 2015
Time: 8:00 AM PDT/11:00 AM EDT

About ScaleX™ Enterprise
ScaleX Enterprise is the enterprise deployment of Rescale’s industry-leading cloud simulation and HPC platform, featuring a unified enterprise simulation environment and a powerful administrative portal, along with direct integrations and management of on-premise HPC resources, schedulers, and software licenses.

A consolidated platform for simulation software and HPC hardware – ScaleX Enterprise enables Fortune 500 CIOs and IT professionals to transform stagnant IT into an agile environment, driving product innovation and providing a competitive advantage.

This article was written by Rescale.