Back to school
The hallway outside the LeBaron auditorium fills with students between classes on Monday, the first day of Iowa State's spring semester. Photo by Bob Elbert.
Q&A: What went wrong (and right) during the online outage
Six minutes after midnight on Jan. 10, in the Administrative Services Building (ASB) second floor data center, a major storage device for Iowa State's servers failed. On its way down, the device sent an electronic SOS to its creator, EMC Corp. About the same time EMC was hearing from the device, Iowa State techs began noticing problems on university sites. It was the start of a 20-hour outage that would take down many sites and services and make for a busy Thursday as information technology services staff and others scrambled to get things back online.
Michael Lohrbach, senior systems analyst in ITS, explains what went wrong (and right) during the outage.
What happened to the storage device?
The storage device has built-in redundancy to prevent failures like this. Unfortunately, both the primary and backup components failed at the same time. That's very rare. Our vendor (EMC) believes there was a bug in the device's software that caused the failure, and a software upgrade is on the way.
When the storage device went down, it knocked out about half of Iowa State's VMware environment. VMware is virtualization technology that allows dozens of servers to be loaded onto a single server for the most cost-effective, efficient handling. In addition to knocking out many servers, the failing device unfortunately caused otherwise functioning machines to lock. The lockups prevented ITS staff from immediately restoring downed servers with backup data.
What sites and services went down in the outage?
Unavailable sites included the ISU homepage, many college and departmental sites, AccessPlus and Blackboard Learn. Microsoft Exchange was running, but only on half the servers it normally uses, causing some delays in message delivery and intermittent connectivity issues.
Were backups available to restore sites?
We have storage and backup devices in both ASB and the Durham Center, and much of the data is replicated between the centers. However, many systems have dependencies, and the level at which they work varies by the method of clustering or failover setup for the service. As we proceeded, decisions needed to be made on restoring the service from backup or working to bring it back online. We ended up using both methods to bring systems online in an effort to restore service as quickly as possible.
Who was working on the outage?
We had about a dozen people working on the issue in the Durham data center. Beyond that room, information technology services staff and college and departmental IT staff elsewhere on campus were working on related tasks. We had a lot of good communication and support from IT units in the colleges and departments, whose staff were very understanding of how these issues happen and willing to help out in any way they could. Four to five EMC engineers were on the case, including Des Moines-area representatives as well as specialists across the world, who were on the phone with us from about 3 a.m. to 7 p.m. Thursday. It was all hands on deck.
With their services and sites out of commission, some departments turned to Facebook, Twitter and low-tech web pages to get information out. How did that work?
We heard comments from people who said they were watching these sites for updates. That was very useful, as was the feedback, relayed through social media sites. We in ITS were focusing on getting things back online, rather than testing systems. It was good to have administrators of other sites and general users keeping us informed about what was and wasn't working.
When was the outage over?
Most sites and services were back online by 6:30 p.m. Thursday. After that, we spent several more hours cleaning up. When something like this happens, systems don't have a chance to shut down gracefully and that can cause issues. So we rebooted systems and took other actions to get them back into a healthy state. By 10 p.m., almost everything was back to normal.
What did the outage cost Iowa State?
Staff working on the outage are salaried P&S staff, and we have full maintenance and support contracts with our vendors. So staffing and repair costs on the failed storage device are covered. It may be hard to tally, but the biggest cost likely incurred in departments that lost functionality for a day. Thankfully, email and file services were functional. Despite the hardships throughout campus, we had a lot of support from the Iowa State community during the outage. We are very appreciative and lucky to have such a good relationship with the IT units and departments across campus and with the people using our services.
What's next?
We're turning our attention to how we can prevent this from happening again and how we can recover more quickly when something goes wrong. We in ITS take our service -- our reputation, security and data protection -- very seriously. We'll continue to work to make that service as good and reliable as it can be.
Leath's statement on Gov. Branstad's proposed FY14 budget
Steven Leath
I join the Board of Regents in thanking Gov. Branstad for his funding recommendations for our three public universities. His recommendations reflect his confidence in the job we are doing in educating thousands of young men and women, in providing valuable research and in extending and applying our knowledge to help Iowa citizens.
The governor’s recommendations directly address growing concerns about the impact college costs have on students and their families. He recommends general fund support that allows us to meet our needs without raising tuition next fall for resident undergraduate students. In addition, he recommends state support for student financial aid, which has not been provided to students in our public universities. The ISU Foundation is working to achieve our goal of generating $150 million in additional private funds over five years for student financial aid.
The governor’s budget recommendation also includes $3.75 million for our biosciences initiative in fiscal year 2014, and full funding of $7.5 million in fiscal year 2015. This is a high priority for Iowa State and for the economic growth of Iowa.
Our ability to bring new technologies to market will be improved with the governor’s recommendation of $3 million in the regent Innovation Fund.
-- President Steven Leath, Jan. 15
Board of Regents response to governor's budget proposal
W-2 changes: A Q&A to help sort out the details
You may have noticed that your 2012 W-2 from Iowa State looks different than it has in the past. Here's a quick Q&A on the change, compiled by Mike Otis, director of human resource services, and Doug Anderson, program manager in the controller's department. Get your 2012 W-2 by logging in to AccessPlus (Employee tab > W-2).
I see a new code in box 12 of my 2012 W-2 -- code DD-Employer SPNSR HLTH CVG and amount. What is this?
The Patient Protection and Affordable Care Act (ACA) requires employers to report the cost of coverage under an employer-sponsored group health plan on an employee’s Form W-2 for 2012. If there is a value to report, it is shown in box 12 with the code DD.
Is this amount included in my taxable income for 2012?
No. The reporting on an employee’s W-2 is for informational purposes only. It is intended to show employees the value of their health care benefits so they can be more informed consumers. The reportable amount includes coverage for the employee and the employee’s beneficiaries and dependents. Only employees, not their beneficiaries and dependents, will receive the information.
Does the reported amount affect my tax liability?
No. The amount reported does not affect tax liability. The value of the employer excludible contribution to health coverage continues to be excluded from an employee's income.
What is not included as "applicable employer-sponsored coverage?"
The IRS has stated that applicable employer-sponsored coverage does not include:
- Dental or vision coverage
- The cost of ISU’s employee assistance program (EAP)
- Long-term care coverage
Does the amount reported in box 12 differ among employees?
Yes. The amount reported in box 12 will depend on the medical plan you are enrolled in (e.g., PPO vs. HMO) as well as the level of your coverage (e.g., single coverage vs. family coverage).
Where can I get more information on this new W-2 reporting requirement?
We encourage employees with questions on IRS tax-reporting requirements to consult with their tax preparer or adviser. You also can review information on the IRS website.
Ombuds helps employees tackle workplace conflicts
Four years after its creation, the ombuds office attracts a steady flow of faculty, staff and graduate and professional students seeking help managing workplace conflicts or other concerns. Since the office opened in September 2008, ombuds officer Elaine Newell has assisted 400 employees and students.
During fiscal year 2012, a record high 104 faculty, staff and graduate and professional students sought assistance at the ombuds office. Here's a look at how the office works and what it can do for you.
Tackle problems before they escalate
Newell's assistance is free, confidential and impartial. The role of the ombuds, she said, is to help people manage conflicts early and informally, before they escalate into formal grievances or litigation.
Newell said interpersonal conflict with supervisors, major professors, or colleagues brings most visitors to the ombuds office. Meetings are usually one-on-one, with Newell and the visitor discussing a situation and developing options. In some cases (about 25 percent in 2012), Newell facilitates direct meetings between individuals in conflict or serves as a go-between for the two parties. She also is available to help supervisors develop options or resources when they’re faced with challenging situations.
"The ombuds office is developing a reputation as a safe place where people can confidentially talk about difficult topics or get ideas for managing interpersonal conflicts," she said.
Online help
The ombuds office partnered with the library to put information about university resources and conflict management in an easy-to-use online package. The Ombuds Office Resource Guide provides links to university offices and policies, as well as to books in the university library about conflict in the workplace, handling tough conversations and dealing with difficult people. Newell said the most popular sites within the guide deal with culture, diversity and conflict; gossip in the workplace and formal resolution processes.
Customized training
For university departments or organizations with specific needs, Newell also will provide customized training or facilitate group discussions about conflict management, respectful communication or other helpful topics.
Ombuds independence
While the ombuds office is part of the president's office, the ombuds office functions independently, Newell said. "As ombuds officer, I'm neither an advocate for my visitors nor a representative for management," she said. "Rather, I'm an advocate for respectful dialogue, fair practices, and mutual understanding."
Miles Lackey, associate vice president and chief of staff, called the ombuds office a valuable resource that's good for the university.
"A lot of people are using it," Lackey said, "and their feedback tells us that they like it. We encourage people to use it. It’s unbiased, it’s confidential, and it can provide resources and ideas about how to handle a difficult situation.”
To visit the ombuds office
Ombuds office hours are Tuesdays, Wednesdays and Thursdays, from 8 a.m. to 5 p.m. in 69 Physics Hall. To make an appointment, call 4-0268 or send an email to ombuds@iastate.edu. Visitors without appointments are welcome to drop in Wednesdays between noon and 1 p.m.
Spring film series is loaded with Oscar nominees
Five of the nine films nominated last week in the Academy Awards' "best picture" category are included in the Cyclone Cinema spring lineup. Chances are good you can enjoy the winner on campus – for free – in the next few months.
Cyclone Cinema is a program of the Student Union Board, with funding provided by the Government of the Student Body. The films are open to the public and admission is free. With the exception of the spring break weekends, films are shown twice a night (7 and 10 p.m.) Thursday through Sunday in the Carver Hall auditorium.
Films are free, the popcorn is not
Cyclone Cinema's concession stand offers popcorn ($2 and $3), boxed candy ($2), bottled water ($1) and Coca-Cola products ($1). You may pay with cash or CyCash at the concession stand. ISU students also may use their Dining Dollars.
Here's a quick look at the spring lineup:
Jan. 17-20: Taken 2 (rated PG-13), Liam Neeson and Maggie Grace
Jan. 24-27: Perks of Being A Wallflower (PG-13), Logan Lerman and Emma Watson
Jan. 31-Feb. 3: Argo* (R), Ben Affleck and Bryan Cranston
Feb. 7-10: Wreck-It Ralph (PG), animated, voices of John C. Reilly and Sarah Silverman
Feb. 14-17: Twilight: Breaking Dawn Part 2 (PG-13), Kristen Stewart and Robert Pattinson
Feb. 21-24: Skyfall 007 (PG-13), Daniel Craig and Javier Bardem
Feb. 28-March 3: Flight (R), Denzel Washington and Don Cheadle
March 7-10: Les Miserables* (PG-13), Hugh Jackman, Anne Hathaway
March 14-15 only: Lincoln* (PG-13), Daniel Day-Lewis and Sally Field
March 16-24: Spring break, no films
March 28-31: This Is 40 (R), Paul Rudd and Leslie Mann
April 4-7: Zero Dark Thirty* (R), Jessica Chastain and Joel Edgerton
April 11-14: The Hobbit: An Unexpected Adventure (PG-13), Martin Freeman and Richard Armitage
April18-21: Gangster Squad (R), Sean Penn and Ryan Gosling
April 25-28: Django Unchained* (R), Jamie Foxx and Christoph Waltz
* Academy Award Best Picture nominee
