Challenges and Opportunities in the HPC Systems Administration and the User Support
Roman Baranowski, University of British Columbia
Venkat Mahadevan, University of British Columbia
In this talk we would like to talk about the current architectural challenges in designing the modern HPC systems. The system administration challenges (access, monitoring, scalability, performance etc...) and approaches and challenges with the user support.
A shared data science platform: bringing power and simplicity to research
Guillaume Moutier, Université Laval
Most of today’s research produce or use data that must be stored, transferred, analyzed, published and preserved. This poses a challenge for many researchers, where local resources are not sufficient to support the required workflow, or offer the necessary level of performance, security and reliability. Moreover, useful research data produced in laboratories are often not easily discoverable and accessible to other researchers.
Université Laval is currently building a service that will help researchers create the most value from this data. It will provide all research domains and teams with the technologies and tools that will enable them to harvest its full potential. This new service will complement the existing resources already available through Calcul Québec and Compute Canada.
The three main objectives of this project are to:
- Build a new Datacenter specialized in data collection, processing and valorization. Adding to the other two main data centers located on different areas of the campus, this Tier-3 facility will allow for scalable and reliable architectures;
- Setup all necessary infrastructures, applications and tools for data storage and processing. With a shared object-based data lake and databases, data discovery, transfer, ingestion, analysis and visualization will be easily made through ready-to-use or on demand data science environments (Hadoop, Spark, VMs, Containers, Notebooks,...);
- Create a service offering for data valorization. A team of data scientists and data engineers will give advice and support for the platform utilization and provide training for its users, and a research data management framework (governance, security, ethics) will enable our researchers to quick-start their data projects along recognized rules and best practices.
This presentation will give an overview of the project, describing the services that will be offered, the different technological infrastructures being deployed (100Gbps networking, network virtualization, object storage, OpenStack and OpenShift environments,…), the tools and applications that are used and the way they are configured (data catalogs, shared notebooks, scalable container-based Spark environments,…). We will also make a quick demonstration of this new integrated environment and show how it simplifies research and allows for new collaboration and interdisciplinary studies, such as the PULSAR project, an innovative and sustainable approach to health.
A Template for Science Web Gateways to Advanced Research Computing: The engine behind the GenAP II Portal
Carol Gauthier, Universite de Sherbrooke
The Genetics and Genomics Analysis Platform (GenAP) is a computing platform for life science researchers across Canada that has been offering services integrated to Compute Canada since 2015. It has been used so far by nearly 200 research groups and is expected to grow with the increasing demand for genomics computing. One of the main GenAP component is a portal gateway allowing users to run their own instances of genomics web applications, which are then linked to the Calcul Quebec Mammouth supercomputer. The development of the second generation of GenAP, expected in production by Spring 2018, will lead to a secure and more flexible Web Gateway that is able to easily leverage several Advanced Research Computing (ARC) resources.
This presentation will mainly focus on the core building blocks and technologies that are used to build the GenAP II Portal Gateway, which have been extracted into a fully functioning and generic Science Web Gateway (SWG) template. This template can then be used as a starting point in the development of any science platforms requiring to leverage ARC resources. The main features of this SWG template includes: 1) Direct connection to the user ACR resources accounts; 2) Management of multiple authentication and identity sources; 3) Groups and projects management; 4) General and easy implementation of ACL; 5) Files management and transfer tools; 6) Management of Web applications through push/pull events from the applications Factory (physical server hosting the application containers); 7) Spawning applications in and/or from a closed secure network; 8) Easy to add new types of web applications.
The software stack and libraries used in the development of this SWG template includes among many others, React and AntDesign on the client side (the browser), and Python3 and Flask on the server side. The template is packaged for a development environment that uses Vagrant, Ansible, and Webpack.
Advancing scientific visualization in Canada
Alex Razoumov, WestGrid
30 minute mini-presentation
WestGrid's and Compute Canada's visualization initiative includes many programs to train and support researchers in data analysis and visualization. Our outreach program features visualization showcases and competitions, full-day in-person training workshops, bimonthly webinars on more advanced and domain-specific topics. Our national visualization team is a venue for HPC support analysts to exchange visualization experience and skills, discuss specific visualization support questions, and learn about new tools. In this talk I will discuss the programs that work, current challenges, and strategies to enhance our visualization offering.
Becoming a National Hosting Site for Research Computing
Martin Siegert, Simon Fraser University
30 minute mini-presentation
Simon Fraser University is the host to Cedar, one of Canada's most powerful academic supercomputers. In 2017, the Cedar facility was ranked one of the top 100 most powerful supercomputers in the world by Top500 and top 100 energy-efficient supercomputers by Green500.
From the design stage of the data centre to operating a large-scale advanced research computing (ARC) facility for Compute Compute Canada (CC): this presentation describes the problems, success stories, and lessons learned during the process of building the Cedar facility at SFU.
Digital Humanities: Who? What? Where?
Megan Lobay, University of British Columbia
This talk will be an introduction to the broad field of Digital Humanities, and how it can be supported at many levels across institutions. Whilst this field can be mistaken for simply computerized English professors, the breadth and scope of research within DH has been expanding within institutions over the past decade or so. Research can take many forms from developing simple tools for text analysis, to using HPC resources to model behaviors and networks across human societies, to developing 3D models of ancient cities and ecosystems. As DH expands into more and more institutions, this talk will provide information about how to recognize Digital Humanities research, data, and working practices. The talk will also look at practical ways that institutions can support the IT needs of Digital Humanities research.
Engaging the Researcher: A Research Support Case Study
Jamie Rosner, University of British Columbia
It’s often not enough to merely offer support to a research community. Such an offering must be discoverable, efficient, and presented in a way that encourages interaction. This presentation will examine a novel support model developed by Compute Canada’s Bioinformatics National Team and delivered to the Canadian bioinformatics community. Attendees can expect to learn the details of this model, the gap it fills, and of course, the challenges and the lessons learned along the way. Was it a success or a failure? Only time will tell.
Federated identities for research
Darren Boss, Compute Canada
30 minute mini-presentation
With thousands of users from dozens of institutions accessing a variety of research infrastructure services and platforms through Compute Canada, managing access and tracking usage is a challenge. In our current system, we must manually vet identities of PIs, collaborators, and lab members who sign up for services by requesting affiliation from their institutions. Federated identities can help, allowing users to log in using existing institutional credentials and automatically providing Compute Canada with institutional metadata critical for user management. There are significant social and technical barriers to implementation of federated identities. If institutions adopt standards such as the Research and Scholarship attribute bundle, implementation of federated identities becomes much simpler and more powerful.
Interacting with Text Annotations: PACTE Platform as a Research and Learning Tool
Pierre André Ménard, Computer research institute of Montreal
Text annotation is a necessary activity in many domains nowadays: researchers use it to identify relevant information, analyze the language level used, link related information to identified concepts, search for linguistic patterns in specialized corpora, enrich texts with new knowledge, create datasets for natural language processing tasks and so on. Teachers can also use text annotations to mark certain sections of a document as important to their student, but can also use them as answers for use-case analysis, critical study of a specialized text or in any way to assess students’ comprehension level. Defining, creating, sharing and managing text annotations for these tasks as either researcher or teacher is challenging because of the lack of available tools.
The Computer Research Institute of Montreal has developed for the past two years an online platform to provide tools for such tasks. This platform, called PACTE (http://pacte.crim.ca), offers researchers and teachers online tools to manage users, annotation schemas, small or large text corpora, lexical resources and collaborative annotation tasks. Programming-inclined research groups or IT departments can also interact with the platform’s secured REST api to upload, download and manipulate annotations and resources, manage user accounts and so on.
The presentation will cover the high-level capabilities and technological aspects of PACTE as well as how it can be used in research or teaching settings. A specialized use case will be demonstrated for each setting, one simulating a text analysis homework and another relating an ongoing machine-learning research project in criminology for young offenders.
Launching a New General Purpose Supercomputer
Greg Newby, Compute Canada
This panel will describe the process from 2014-present of being selected to host one of Canada’s largest supercomputers, in support of thousands of users across all scientific disciplines. The CFI’s Challenge 2 program provided capital, with matching and engagements from hosting sites and provinces, to launch a huge refresh of Canada’s national advanced research computing platform. The process began with selecting hosting sites for the systems, and proceeded through a series of agreements and planning. A national procurement team matched user needs with system specifications, and purchased new systems, storage, networking, and other components. At the same time, the organizational structure of Compute Canada evolved from independent hosting sites to a set of interdependent teams spanning Compute Canada members and partners. The resulting systems have been in operation for a year, and have hosted thousands of computational campaigns. It all looks easy, when it works. This talk will share some of the insights gained, challenges encountered, and tremendously beneficial outcomes of the effort.
Strategies for Supporting Advanced Research Computing
Mark Roman, Simon Fraser University
Dugan O'neil, Simon Fraser University
This panel will be moderated by the CIO of SFU and include insight from several institutions on their strategies for supporting the computing and storage needs of their researchers beyond traditional university IT support.
The panel will share what has worked for their institution in terms of reporting structures, creating teams, shared staffing, partnerships with other institutions, best practices, etc.
Supporting Research Platforms & Portals
Marc Rousseau, Compute Canada
Research platforms and portals (RPPs) provide a Web-based gateway to data and functionality. The Compute Canada national platform for advanced research computing hosts over 50 RPPs, spanning all scientific disciplines and interests. Some portals serve a small number of researchers or research groups, while others handle thousands of users around the world. This session will provide an overview of current RPPs, and the national platform’s resources for building and maintaining them. Panelists will describe their own platforms, and what they do to keep them active and vibrant. Attendees will learn about available tools and services from Compute Canada, the variety of platforms and services in Canada, and how to approach building a new RPP. Audience Q&A and discussion will be encouraged.