Rescale is pleased to introduce a powerful new feature now available on our platform: In-Browser SSH. For jobs using Linux compute clusters on supported providers, you will now see an SSH panel under the Live Tailing panel that allows you to connect to the running cluster using SSH:

The new In-Browser SSH connection panel

For jobs running on supported hardware, this will be available even if you did not set up an SSH public key in your job settings. However, for maximum flexibility, we still recommend setting it up if you anticipate a need for SSH connectivity.

There are two ways to launch an In-Browser SSH session. Clicking the link itself will open a tab directly in the SSH panel, while clicking the pop-out icon to the right will create new browser window/tab with the SSH session maximized.

Click the IP link (boxed in red) to open an In-Browser SSH panel in the SSH Sessions tab. Click the pop-out link (circled in blue) to open a session in a new browser window/tab

Opening new SSH sessions in tabs will open the tab directly in the SSH Sessions panel. ANSI colors are supported.

What’s more, if you have a multi-node cluster, you will see a connection link to each node in the cluster. This is tremendously useful if you are trying to quickly connect to several different nodes within the cluster.

A connection entry will appear for each separate node in the cluster.

Note that currently, the active SSH tabs will only persist while the Job Status view is active. In other words, switching away from the Status view to, for example, the Results view and then back will close the SSH panels. If you need to persist the SSH session, we recommend launching a session in a separate browser window or tab.

Since the In-Browser SSH terminal uses a HTML canvas element, copy-and-pasting must go through an intermediate clipboard. To open it, click the small tab near the right-top corner of the SSH window. Anything inside the text box in this side panel will be available to the SSH session. Then paste it into the SSH terminal using Ctrl+Shift+V on Windows or Command+V on a Mac.

Click the tab near the right-top corner of the in-browser SSH panel to toggle the clipboard pane

Note that the SSH support, going through canvas, means that there is no graphical device available. In other words, this is text-mode only without X11 forwarding. If you need X11 forwarding, you must use your OS’s native terminal with appropriate X11 support installed.

Also note that the in-browser SSH panel will only be available to owners of the job. If you share the job using the job-sharing functionality, the recipient will not automatically get SSH access to the cluster.

This article was written by Alex Huang.

Rescale is pleased to introduce a handy new feature now available on our platform, In-Browser Desktops. With this new feature, all you need is a modern web browser and a few clicks to connect to a powerful remote desktop for your pre/post-processing and visualization needs.

In-Browser Desktops are the fastest way to launch a visualization node from a running job—no need to set a password, download a connection file, or run a tunnel script. This also means that you will be able to securely access the remote desktop even if you are behind a firewall over simple HTTPS. Everything runs seamlessly and securely through the browser.

When connecting to a desktop, you will be able to choose from using an In-Browser Desktop or downloading a Remote Desktop Protocol (RDP) or Virtual Network Computing (VNC) client file to use your own remote desktop client, just as before.

Here is how to launch an In-Browser Desktop for visualization from a running job.

Step 1: Select “Visualize” for a running job.

Launching an In-Browser Desktop: Step 1

Step 2: Select “Launch” to confirm your Desktop configuration.

Launching an In-Browser Desktop: Step 2

Step 3: Click “Connect using In-Browser Desktop” and…

Launching an In-Browser Desktop: Step 3

…Voila! You can now monitor your jobs directly from your browser, as shown in the STAR-CCM+ example below.

Launching an In-Browser Desktop: Step 4

Compare this with using your own remote desktop client to visualize your results, which is a multi-step process as outlined below.

Step 1: Download an RDP client specific to your operating system.

Step 2: Select Generate Password and copy the password to your clipboard.

Step 3: Download an RDP client file.

Step 4: Launch your RDP client and open the file downloaded in Step 3.

Step 5: Wait for the system to initialize, provide an appropriate username and password from Step 2 and log-in.

This article was written by Alex Huang.

As a long-time GNU Emacs user, I have come to take C-x C-e (eval-last-sexp) for granted. That is, in almost any file that I’m editing, I can insert a parenthesis expression wherever I am, hit C-x C-e, and get a result displayed in the minibuffer (or if programming in a lisp, in the REPL).

Many other languages provide console interfaces or REPLs, but they usually come as separate features of the language, rather than an assumed mode of interaction. I’ll admit that this is a superficial observation, but one could reframe this terms of a “script-centric” or a “REPL-centric” approach to user interaction. Imagine yourself learning a new programming language. A “script-centric” approach would quickly introduce you how to run a hello world program from a terminal (or IDE), whereas a “REPL-centric” approach would quickly introduce you how to drop into a REPL and evaluate the requisite expressions to achieve the hello world effect.

In other words, a “script-centric” approach conceptually separates the editing and evaluation cycles, whereas the “REPL-centric” approach is designed around a single flow of expression, evaluation, and feedback. Each method has its strengths and weaknesses, but for prototyping and experimentation, it’s hard to beat the feeling of “being with the code” from evaluating code right at the cursor. In Emacs, for languages in the lisp family, this is generally the expected method of interaction.

How might we demonstrate this? Well, my colleague Mark wrote a tutorial on setting up a Keras training job back in February. The software has been updated since then, so we’ll create an updated version of that job; but instead of raw python, we’ll use hylang, which is basically a lisp for the Python interpreter.

hylang

Assuming you have a recent version of Emacs running (say, version 24 and up) with the Emacs package manager set up, you would want to install hy-mode (M-x package-install RET hy-mode RET), and one of the many lispy paren packages, e.g. paredit, smartparens, lispy.

In our new file, cifar10_cnn.hy, make sure you are on Hy mode. Then use M-x inferior-lisp to start a hy repl! Scroll to the bottom for a video demonstration of Emacs interaction.

1 First let’s take care of the imports 

1.1 Python

1.2 Rewriting this in hy:

51 words to 39. Not bad for succinctness!

2 Then let’s rewrite the constants

2.1 Python

2.2 Hy

Hylang takes inspiration from Clojure, but unlike Clojure, allows multiple variable def-s by default. In this case it’s quite handy.

3 The functions

Note that the input_shape argument ordering has changed since the previous version from (channels, rows, columns) to (rows, columns, channels)

3.1 Python

3.2 Hy

In the code below, we could have written the variable destructuring for TRAIN and TEST with a single step like so:

(let [ [[X-train y-train] [X-test y-test]] (cifar10.load_data) ] )

Notice how using doto here really simplifies repetitive assignment. Since expressions evaluate to the final form inside them, which gets passed through by doto, our model setup in make-network doesn’t even require a variable name.

Hylang also supports the handy threading macros (->) a la clojure, which makes it easy to chain statements together; in this example we just thread the format string through the .format function, then print it.

4 The main runner block

4.1 Python

4.2 Hy

5 A quick video

In this short video you can see how one might load the resulting hylang file into emacs and run it through the hy REPL, evaluating the statement on point (which show up as ^X ^E in the key overlay), and otherwise switching back and forth between the REPL and the edit buffer.

Towards the end, you can see how I missed the numpy import, added the import to the (import ...) form, and re-evaluated the entire import form. I moved the cursor to the end of the form and used eval-last-sexp for evaluate the expression, but could have also used Ctrl Alt x or lisp-eval-defun, which would evaluate the top level expression surround the cursor.

Since load-dataset and make-network take no arguments, it is convenient to wrap the function bodies in a let or doto expression (or form), repeatedly evaluating the block at-point, checking the REPL output, and when satisfied, wrap the let block into a defn form.

6 Running it on the platform

The video stops right after I start the actual model training, because my computer doesn’t have a good GPU, and the training would take a long time. So instead we’ll put it back on the Rescale platform to complete, first installing the hy compiler by running

then simply calling hy cifar10_cnn.hy to replicate Mark’s previous example output.

(note that in this example, we’re running the latest version (version 0.11.0, commit 0abc218) of hylang directly from github). If you installed hy using pip install hy it may be using an older assignment let/with syntax.

A ready-to-run example can be found here. In addition, a tanglable version of this post can be found on github.

7 Pardon the pun

But is it hy time? As you can see, the REPL integration into Emacs has its quirks. Debugging errors from deep stack traces is also a bit more challenging due to less support from mature debugging tools. But if you enjoy working with lisps and want or need to work with the python ecosystem, hy is a functional, powerful, and enjoyable tool to use, for web development, machine learning, or plain old text parsing. Did I mention you can even mix and match hy files and py files?

This article was written by Alex Huang.