Mini-course: Introduction to Python — Sept. 11-14

By | Data, Educational, Events, General Interest, News | No Comments

Asst. Prof. Emanuel Gull, Physics, is offering a mini-course introducing the Python programming language in a four-lecture series. Beginners without any programming experience as well as programmers who usually use other languages (C, C++, Fortran, Java, …) are encouraged to come; no prior knowledge of programming languages is required!

For the first two lectures we will mostly follow the book Learning Python. This book is available at our library. An earlier edition (with small differences, equivalent for all practical purposes) is available as an e-book. The second week will introduce some useful python libraries: numpyscipymatplotlib.

At the end of the first two weeks you will know enough about Python to use it for your grad class homework and your research.

Special meeting place: we will meet in 340 West Hall on Monday September 11 at 5 PM.

Please bring a laptop computer along to follow the exercises!

Syllabus (Dates & Location for Fall 2017)

  1. Monday September 11 5:00 – 6:30 PM: Welcome & Getting Started (hello.py). Location: 340 West Hall
  2. Tuesday September 12 5:00 – 6:30 PM: Numbers, Strings, Lists, Dictionaries, Tuples, Functions, Modules, Control flow. Location: 335 West Hall
  3. Wednesday September 13 5:00 – 6:30 PM: Useful Python libraries (part I): numpy, scipy, matplotlib. Location: 335 West Hall
  4. Thursday September 14 5:00 – 6:30 PM: Useful Python libraries (part 2): 3d plotting in matplotlib and exercises. Location: 335 West Hall

For more information: https://sites.lsa.umich.edu/gull-lab/teaching/physics-514-fall-2017/introduction-to-python/

 

Info sessions on graduate studies in computational and data sciences — Sept. 21 and 25

By | Educational, Events, General Interest, News, Research | No Comments

Learn about graduate programs that will prepare you for success in computationally intensive fields — pizza and pop provided

  • The Ph.D. in Scientific Computing is open to all Ph.D. students who will make extensive use of large-scale computation, computational methods, or algorithms for advanced computer architectures in their studies. It is a joint degree program, with students earning a Ph.D. from their current departments, “… and Scientific Computing” — for example, “Ph.D. in Aerospace Engineering and Scientific Computing.”
  • The Graduate Certificate in Computational Discovery and Engineering trains graduate students in computationally intensive research so they can excel in interdisciplinary HPC-focused research and product development environments. The certificate is open to all students currently pursuing Master’s or Ph.D. degrees at the University of Michigan.
  • The Graduate Certificate in Data Science is focused on developing core proficiencies in data analytics:
    1) Modeling — Understanding of core data science principles, assumptions and applications;
    2) Technology — Knowledge of basic protocols for data management, processing, computation, information extraction, and visualization;
    3) Practice — Hands-on experience with real data, modeling tools, and technology resources.

Times / Locations:

HPC training workshops begin Thursday, Sept. 21

By | Educational, Events, General Interest, HPC, News | No Comments

series of training workshops in high performance computing will be held Sept. 21 through Oct. 31, 2017, presented by CSCAR in conjunction with Advanced Research Computing – Technology Services (ARC-TS). All sessions are held at East Hall, Room B254, 530 Church St.

Introduction to the Linux command Line
This course will familiarize the student with the basics of accessing and interacting with Linux computers using the GNU/Linux operating system’s Bash shell, also known as the “command line.”
Dates: (Please sign up for only one)
• Thursday, Sept. 21, 9 a.m. – noon (full descriptionregistration)
• Thursday, Sept. 28, 9 a.m. – noon (full description | registration)
Location:
East Hall, Room B250, 530 Church St.

Introduction to the Flux cluster and batch computing
This workshop will provide a brief overview of the components of the Flux cluster, including the resource manager and scheduler, and will offer students hands-on experience.
Dates: (Please sign up for only one)
• Thursday, Sept. 28, 1 – 4 p.m. (full description | registration)
• Monday, Oct. 2, 9 a.m. – noon (full description | registration)
Location:
East Hall, Room B254, 530 Church St.

Advanced batch computing on the Flux cluster
This course will cover advanced areas of cluster computing on the Flux cluster, including common parallel programming models, dependent and array scheduling, among other topics.
Dates: (Please sign up for only one)
• Tuesday, Oct. 10, 1 – 5 p.m. (full description | registration)
• Thursday, Oct. 12, 9 a.m. – noon (full description | registration)
Location:
East Hall, Room B254, 530 Church St.

Hadoop Workshop
Learn how to process large amounts (up to terabytes) of data using SQL and/or simple programming models available in Python, Scala, and Java.
Date:
• Tuesday, Oct. 31, 1 – 5 p.m. (full description | registration)
Location:
East Hall, Room B254, 530 Church St.

Flux HPC Blog: Querying data with SparkSQL

By | Data, General Interest, HPC, News | No Comments

SparkSQL is a way for people to use SQL-like language to query their data with ease while taking advantage of the speed of Spark, a fast, general engine for data processing that runs over Hadoop. I wanted to test this out on a dataset I found from Walmart with their stores’ weekly sales numbers. I put the csv into our cluster’s HDFS (in /var/walmart) making it accessible to all Flux Hadoop users.

U-M, SJTU research teams share $1 million for data science projects

By | Data, General Interest, Happenings, News, Research | No Comments

Five research teams from the University of Michigan and Shanghai Jiao Tong University in China are sharing $1 million to study data science and its impact on air quality, galaxy clusters, lightweight metals, financial trading and renewable energy.

Since 2009, the two universities have collaborated on a number of research projects that address challenges and opportunities in energy, biomedicine, nanotechnology and data science.

In the latest round of annual grants, the winning projects focus on data science and how it can be applied to chemistry and physics of the universe, as well as finance and economics.

For more, read the University Record article.

For descriptions of the research projects, see the MIDAS/SJTU partnership page.

SAVE THE DATE: MIDAS Annual Symposium, Oct. 11

By | Events, General Interest, News | No Comments

Please join us for the 2017 Michigan Institute for Data Science Symposium.

The keynote speaker will be Cathy O’Neil, mathematician and best-selling author of “Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy.”

Other speakers include:

  • Nadya Bliss, Director of the Global Security Initiative, Arizona State University
  • Francesca Dominici, Co-Director of the Data Science Initiative and Professor of Biostatistics, Harvard T.H. Chan School of Public Health
  • Daniela Whitten, Associate Professor of Statistics and Biostatistics, University of Washington
  • James Pennebaker, Professor of Psychology, University of Texas

More details, including how to register, will be available soon.

New Data Science Computing Platform Available to U-M Researchers

By | General Interest, Happenings, HPC, News | No Comments

Advanced Research Computing – Technology Services (ARC-TS) is pleased to announce an expanded data science computing platform, giving all U-M researchers new capabilities to host structured and unstructured databases, and to ingest, store, query and analyze large datasets.

The new platform features a flexible, robust and scalable database environment, and a set of data pipeline tools that can ingest and process large amounts of data from sensors, mobile devices and wearables, and other sources of streaming data. The platform leverages the advanced virtualization capabilities of ARC-TS’s Yottabyte Research Cloud (YBRC) infrastructure, and is supported by U-M’s Data Science Initiative launched in 2015. YBRC was created through a partnership between Yottabyte and ARC-TS announced last fall.

The following functionalities are immediately available:

  • Structured databases:  MySQL/MariaDB, and PostgreSQL.
  • Unstructured databases: Cassandra, MongoDB, InfluxDB, Grafana, and ElasticSearch.
  • Data ingestion: Redis, Kafka, RabbitMQ.
  • Data processing: Apache Flink, Apache Storm, Node.js and Apache NiFi.

Other types of databases can be created upon request.

These tools are offered to all researchers at the University of Michigan free of charge, provided that certain usage restrictions are not exceeded. Large-scale users who outgrow the no-cost allotment may purchase additional YBRC resources. All interested parties should contact hpc-support@umich.edu.

At this time, the YBRC platform only accepts unrestricted data. The platform is expected to accommodate restricted data within the next few months.

ARC-TS also operates a separate data science computing cluster available for researchers using the latest Hadoop components. This cluster also will be expanded in the near future.

XSEDE Research Allocation Requests due July 15th

By | Educational, General Interest, HPC, News | No Comments

XSEDE Allocations award eligible users access to compute, visualization, and/or storage resources as well as extended support services.

XSEDE has various types of allocations from short term exploratory request to year long projects. In order to access to XSEDE resources you must have an allocation. Submit your allocation requests via the XSEDE Resource Allocation System (XRAS) in the XSEDE User Portal.

ARC-TS consultants can help researchers navigate the XSEDE resources and process. Contact them at hpc-support@umich.edu

Big Data in Transportation and Mobility symposium highlights diverse, emerging issues

By | Events, General Interest, News | No Comments

MBDH-transThe Big Data in Transportation and Mobility symposium held June 22-23, 2017, in Ann Arbor, MI brought together more than 150 data science practitioners from academia, industry and government to explore emerging issues in this expanding field.

Sponsored by the NSF-supported Midwest Big Data Hub (MBDH) and the Michigan Institute for Data Science (MIDAS), the symposium featured lightning talks from transportation research programs around the Midwest; tutorials and breakout sessions on specific issues and methods; a poster session; and a keynote address from two representatives of the Smart Columbus project: Chris Stewart, Ohio State University Associate Professor of Computer Science and Engineering, and Shoreh Elhami, GIS Manager for the city of Columbus.

Speakers and attendees came from a number of organizations from across the midwest including the University of Michigan, University of Illinois, University of Nebraska, University of North Dakota, North Dakota State University, Ohio State University, Purdue University, Denso International America, Fiat Chrysler, Ford Motor Company, General Motors, IAV Automotive Engineering and Yottabyte.  

“This was an extremely valuable opportunity to share information and ideas,” said Carol Flannagan, one of the organizers of the symposium and a researcher at MIDAS and the U-M Transportation Research Institute. “Cross-discipline and cross-institutional collaboration is crucial to the success of Big Data applications, and we took a significant step forward in that vein during this symposium.”

Topics addressed in talks, breakouts, and tutorials included:

  • New Analytic Tools for Designing and Managing Transportation Systems
  • New Mobility Options for Small and Mid-sized Cities in the Midwest
  • Automated and Connected Vehicles
  • Transforming Transportation Operations using High Performance Computing
  • On-Demand Transit
  • Using Big Data for Monitoring Bridges

At the closing session, participants outlined some areas that could be fruitful to focus on going forward, including increasing data-science literacy in the general public; diversity and workforce development in data science; public data-sharing platforms and partners; and privacy issues.

For a complete list of speakers and topics, please see the agenda. Videos of selected talks will be posted at midas.umich.edu in the coming days.

MICDE announces 2017-2018 Fellowship recipients

By | Educational, General Interest, Happenings, News | No Comments

MICDE is pleased to announce the recipients of the 2017-2018 MICDE Fellowships for students enrolled in the PhD in Scientific Computing or the Graduate Certificate in Computational Discovery and Engineering. We had 91 applicants from 25 departments representing 6 schools and colleges. Due to the extraordinary number of high quality applications we increased the number of fellowships from 15 to 20 awards. See our Fellowship page for more information.

AWARDEES

Diksha Dhawan, Chemistry
Negar Farzaneh, Computational Medicine & Bioinformatics
Kritika Iyer, Biomedical Engineering
Tibin John, Neuroscience
Bikash Kanungo, Mechanical Engineering
Yu-Han Kao, Epidemiology
Steven Kiyabu, Mechanical Engineering
Christiana Mavroyiakoumou, Mathematics
Ehsan Mirzakhalili, Mechanical Engineering
Colten Peterson, Climate and Space Sciences & Engineering
James Proctor, Chemical Engineering
Evan Rogers, Biomedical Engineering
Longxiu Tian, S. Ross School of Business
Jipu Wang, Nuclear Engineering and Radiological Sciences
Yanming Wang, Chemistry
Zhenlin Wang, Mechanical Engineering
Alicia Welden, Chemistry
Anna White, Industrial & Operations Engineering
Chia-Nan Yeh, Physics
Yiling Zhang, Industrial & Operations Engineering

HONORABLE MENTIONS

Geunyeong Byeon, Industrial & Operations Engineering
Ayoub Gouasmi, Aerospace Engineering
Joseph Kleinhenz, Physics
Jia Li, Physics
Changjiang Liu, Biophysics
Vo Nguyen, Computational Medicine & Bioinformatics
Everardo Olide, Applied Physics
Qiyun Pan, Industrial & Operations Engineering
Pengchuan Wang, Civil & Environmental Engineering
Xinzhu Wei, Ecology & Evolutionary Biology