Network Outage Announcements
Unauthorized Network Intrusion
Now that all network systems and servers are back online, I wanted to provide a final update on the campus’ recovery from an unauthorized network intrusion we discovered on March 5, 2021.
No remote computers or laptops were affected. Almost all desktop computers on campus had up-to-date anti-virus software that worked.
To date, we have no indication that data was exfiltrated nor that personally identifiable data was compromised.
All administrative servers were backed up four hours before the incident, and all of them have been restored and were back online earlier this week.
If you are experiencing any issues accessing applications or servers, contact the HelpDesk at 915-747-HELP (4357) or firstname.lastname@example.org. You may need to update the way you access some of these network resources.
A very small number of faculty were impacted by research servers or computers that were damaged and did not have recent backups. I have met personally with those faculty to address their specific situations.
Although we expect a report in the next couple of weeks, we may never know exactly where the intrusion occurred or how it happened.
Every month, like other institutions of our size and scope, our Information Resources team successfully mitigates a very large number of software vulnerabilities and fends off about 20,000 external security threats to our network. We are continuing to take steps to enhance the security of our systems in the face of evolving cybersecurity threats.
Our next step will be to review all of our systems and prepare for new federal requirements related to cybersecurity for research universities that we know are coming. We plan to involve a cross-campus working group in this process, including faculty and staff with relevant expertise. We may also use an outside consulting firm to bring expertise from elsewhere and help guide our work.
Thank you again for your patience.
Additional Network Services Restored, On-Campus Work Can Resume Monday
The University’s Information Resources team has continued to work to recover central systems and applications this week. Access has been restored to my.utep.edu, Goldmine, the Library’s online resources, and Jabber, a service that allows University phone lines to be answered remotely.
The Pete’s Payment Options portal is available to process online payments. Students who had payment deadlines during the outage will not have late fees assessed to their accounts. For payment-related questions, please email email@example.com.
The team is finishing scans on campus computers. Faculty and staff who were previously working on campus but were instructed to work remotely due to the outage may return to campus on Monday, March 22, 2021.
As a reminder, if you are using a UTEP computer to work remotely, software called Carbon Black will need to be installed on that machine. Please follow this link to open a service request with Technology Support to begin the process. After you log in, you will be asked to type in the UTEP tag number of the computer and then select “Add Request.” If you have more than one UTEP computer at home, please click the link again to submit additional requests. You should complete this process for any UTEP desktop computers and laptops. For questions about this process, please refer to these FAQs.
We appreciate your patience and cooperation as we continue to bring all systems and applications back online.
Update for March 15, 2021
The University’s Information Resources team continued to recover central systems and applications over the weekend. Today, campus teams started checking every computer on campus.
The University continues to look for evidence of any personal information being compromised but so far has not found any evidence of that.
Update for March 12, 2021
All email accounts have been restored, as well as webmail.
We also confirmed that the University back-up servers have not been compromised. PeopleSoft and payroll have not been affected.
In addition, UTEP’s Proactive COVID Testing Program will resume on Monday, March 15. Please check utep.edu/covidtesting for times.
Over the weekend and into next week, our efforts will be on restoring on-campus Wi-Fi, Goldmine, Pete’s Payment Options, and other critical core systems. Technology Implementation Managers will also be working with deans and departments to prioritize work.
As our Information Resources team continues to work on restoring all systems, please work from home through next Tuesday and do not make any changes to campus computers until we provide another update that it is safe to do so.
Our next update will be provided on Monday.
Update for March 11, 2021
Today, faculty and staff email came back online, and we restored Wi-Fi in the student housing communities. The team was able to recover the email accounts back to Friday morning, March 5. Any emails sent to your accounts between Friday and when the email server was restored today may not be received.
The team is still working to restore the Wi-Fi for the rest of the campus, my.utep.edu, Goldmine, Pete’s Payment Options, and other critical systems.
Student Business Services will not be open to process cash transactions on Friday, March 12, 2021. The online Pete’s Payment Options is unavailable until further notice. As a result, payment due dates have been extended from March 15 to March 22, 2021. If you have questions, please send them to firstname.lastname@example.org.
As we continue to restore services in the coming days, please continue to work from home and refrain from turning on or making any changes to campus computers until we provide another update that it is safe to do so.
We will continue to provide updates here as new services are back online.
Email is Being Restored
9:30 a.m. March 11
Miners - UTEP email is coming back up! Some faculty and staff may already have access through Outlook, and others should regain access throughout the day. Email may not be available from your phone immediately.
Update for March 10, 2021
Earlier today, the UTEP website was restored. Not all services are yet available, but information on the outage can be found at https://www.utep.edu/newsfeed/network-outage-announcement.html.
The Office of International Programs has a Blackboard room open during business hours and is also providing updates through Facebook at www.facebook.com/oipUTEP.
The University will make necessary extensions for seniors in their final semester to obtain their clearances to graduate.
The UTEP COVID-19 Vaccination Program has not been impacted by the outage. 1,123 individuals were vaccinated on Friday and another clinic will be hosted later this week.
The Information Resources team is working through the night with Microsoft to restore email as soon as possible. In addition, they are working on restoring Wi-Fi at student housing as a priority.
The Administrative Forum scheduled for Thursday has been postponed. As soon as our systems and people can support it, we will be hosting a town hall. The priority at the moment is to let the experts focus on restoring our systems.
Non-essential personnel are asked to work from home through Thursday and Information Resources has asked that you not come to campus to turn on any computers in your offices at this time.
We will post updates on the UTEP website and social media accounts. Thanks again for your patience and support of our Information Resources team.
Update for March 9, 2021
7:34 p.m. March 9
Today our Information Resources team continued to work on restoring the exchange server, which houses the University’s email. Blackboard and Banner have been up and running and accessible.
We have continued to look and are not aware of any personal information being compromised.
Non-essential personnel should continue to work from home through Wednesday.
Update for Monday, March 8, 2021
9:47 p.m. March 8
Our team has continued to work on prioritizing and restoring our systems. Earlier today, Blackboard became operational and online classes were able to resume as scheduled. In addition, throughout the day we were able to restore Banner, which is now available.
The Library and Student Recreation Center remained open and will continue to do so.
At this time, we ask that non-essential personnel continue to work remotely through Tuesday. Updates will be provided each day and can be found on our social media accounts.
Network Outage Update
9:35 p.m. March 7
Early Friday morning, UTEP detected an unauthorized and potentially malicious intrusion in our on-campus network.
Following our standard procedures, we immediately turned off all of our campus systems and have been working throughout the weekend to test and bring each system back online after checking it thoroughly.
We have been checking diligently and we are not aware of any personal information that has been compromised. Of course, we will continue to look for this in the coming days.
We are prioritizing restoring our online learning and teaching systems first. We are trying to have Blackboard up and running beginning Monday morning so that all online classes can resume. Faculty should work with students and adjust deadlines and coursework accordingly. Because faculty and staff email continues to be down, faculty and students should communicate via Blackboard regarding any questions about in-person classes.
We are also continuing to work to safely bring all of our other systems online. This is a time-consuming process because every machine on campus must be checked. We will accomplish this as quickly and safely as possible.
All non-essential faculty, staff and students should work from home Monday. Remote computer systems are operational. Essential personnel will be required to report to work on campus. Our walk-up COVID-19 testing sites will be closed at this time. The state drive-through testing site is still available. On-campus athletic practices will continue as scheduled.
We will share additional information as systems are checked and turned back on. Because email and the server hosting the UTEP website were affected, we will post information on UTEP social media accounts.
Thank you for your patience.
Network Outage Update
1:51 p.m. March 7
Miners - some of you have reported that fellow classmates have had brief access to UTEP network systems. As the team continues to make progress on the network issues, some systems may come back up temporarily for some users. When they are all back up and ready to be accessed, we will make an announcement.
Thank you for your continued patience.
Network Outage Update
1:26 p.m. March 6
Miners - As you know, the University had a technical issue yesterday. Out of an abundance of caution, we turned off some computing systems and our IT department is continuing to work to identify and resolve the issue. Each system is being tested and will come up one at a time. This is a time-consuming process. We will continue to provide more information as it becomes available.
We understand that our students have concerns about missed exams and assignment deadlines. Faculty will be flexible regarding these issues, and any exams affected will be rescheduled.
Students who live on campus: If you are having trouble accessing the internet, you may check out a free Wi-Fi hotspot from the front desks of each housing community starting at 5 p.m. today. One hotspot will be available per apartment. When the front desk is closed, you may contact the RA on duty to check out a hotspot.
9:53 a.m. March 5
Miners - UTEP is experiencing campuswide network issues. This is affecting access to email and other campus systems. It is also affecting our COVID-19 walk-up testing sites, which are now closed for the day.
If you would like to receive a COVID-19 test today, you may access the UTEP lane at the state drive-through testing site in the Schuster 3 parking lot. Today's vaccine clinic will take place as scheduled.
IT is working to resolve these issues as quickly as possible. We will provide updates here as new information becomes available.