Aging server kills campus net passwords
By Susan Walley
Senior Staff Writer
|Photo by Tashieka Hammond
UCIS tech Kristen Anderson assists student Terrance Curry with login problems.
A significant number of students and faculty were prevented from accessing their webmail and blackboard accounts on Jan. 24 and 30 due to ongoing infrastructure updates of the University’s network system, which are the beginning stages of a total infrastructure upgrade UCIS has begun planning for the fall.
According to a statement released by University Computing and Information Services (UCIS), early in the morning of the Jan. 24 the server responsible for storing login information for all network users began expiring passwords.
Before 8 a.m., UCIS discovered that the server was running a process and stopped it, said Associate Vice Chancellor for Information Resources and CIO Robert Orr.
At the time, Orr said, they were unaware of what the program had done. It was not until 8:10 a.m. that the UCIS helpdesk informed operations that they had already received a large volume of calls reporting login failures.
Beginning at 8:15 a.m., UCIS began engaging in conference calls and briefings in order to alert campus and began planning to fix the damage, Orr said.
After the problem was identified, Orr began contacting faculty by email through the listserv to alert them to the event. This process began at 8:36 a.m., but at this time, according to Orr’s message, the UCIS team was still “troubleshooting” to find a solution. It was not until later that UCIS discovered the solution to the problem was the resetting of the user’s password.
Immediately after isolating the problem, work began on a script that would allow users to reset their passwords themselves, as initially users had to rely on the helpdesk and library computer lab to have their passwords changed.
By Thursday evening, the script was added to the UNCP homepage, webmail, blackboard, Clean Access Agent and Braveweb.
According to Maureen Windmeyer, director of Client Service for UCIS, Thursday night thousands of users were able to reset their own passwords successfully.
The reset process, Windmeyer explained, utilized either the security question set by the user or the user’s date of birth. However, the reset was limited to one use, to prevent others from altering your password, she said.
According to an email sent to the faculty listserv by Robert Orr, the creation of a security question for password resetting was launched only last year, so it was necessary to include a date of birth option to allow all users to reset their passwords.
As those who had been affected by the problem were unable to receive these email updates, UCIS broadcast a voice mail to all faculty members once webpage instructions for self-resetting were in place.
The incident was later found to be connected to the upgrading process currently taking place within UCIS.
The servers that UNCP’s networks operate on have become old and outdated. Windmeyer said the servers Snappy and Harmony, which store information for students and faculty, have been in use since prior to 1999.
Orr said that in order to run more updated processes and programs, the older servers will require patches. Yet as these patches are installed, they take up space within the server and require more processes to occur internally which may slow the server down.
According to the statement released by UCIS, patches for the new installation of the Active Directory System had recently been installed on the authorization server, later causing the password resetting process.
“...sometimes those things happen no matter how careful you are.” Orr said.
According to Windmeyer, as of Jan. 31 about 1,000 users who had logged in to the system after Jan. 1 still had not reset their passwords.
Yet Windmeyer said many of these may be recent graduates or students who have otherwise left the University.
On the morning of Jan. 30, network users again experienced a network interruption connected to the new installations on the authentication server. The patches, according to Orr, slowed the server down to the extent that to the end user, it appeared the server was not functioning at all.
In response to this problem, UCIS launched a new server into operation on Jan. 31 at 1 p.m.
Typically, such a switch over would occur in the early hours of the morning, Windmeyer said, but it was decided that it would be best to launch the new server during a time when the system was fully operational and staff was present to watch for problems.
This new server, according to Orr, initially took over authorization for Braveweb only. Orr said that once the server proves to have taken over Braveweb’s authorization without issue, webmail and later blackboard, will be switched over as well.
Orr alerted faculty via the listserv prior to the server switchover.
He said that “we think we can make these changes without impacting users,” but added that if the switch did not go smoothly as hoped “early morning classes that use technology have been impacted twice. We are concerned about additional impacts on student learning if something did not go as planned overnight.”
Orr said that the new server will help to remove stress on the old authentication server, hopefully preventing future interruptions from occurring.