Easy-speed run 47 hours to restore the secret

The main data center was destroyed, most of the staff were killed. 47 hours to resume trading! 911 events during the year, easy to speed the company did. In the same case, Chinese enterprises can do it? Current events overseas in 911 cases chosen the easy to speed losses, and hopes to give some reference to China's enterprises.

8:00 am on September 11, 绾︾憻澶缁磋? Luo (Joseph Noviello) has been very good mood. He received information from the travel company, even though the sea climate is not very good, but the annual fishing trip can still continue. Personally think of beautiful beaches and fishing to the stimulus, he danced up happy.

But not long after, a television screen to play ignorant Noviello, and began his life's most intense two days. From his apartment in Manhattan to see the TV, a plane hit the center of a landmark World Trade Center - his office and the company's main data center is there.

Noviello is Kantefeici Fitzgerald (Cantor Fitzgerald) financial services company affiliated trade speed (eSpeed) software company, technical director of electronic secondary trading. Easy-speed running a company dedicated trading network, the network is connected to more than 700 financial institutions. Noviello's office is on the World Trade Center North Tower 103 level, is the center of the aircraft impact floors, easy fast becoming one of its most costly.

However, eight in the morning on September 13, when the bond market to re-open, easy to speed back to normal operation, this is only after a disaster just over the past 47 hours. Detailed planning, help from other companies and an indomitable team spirit so easy to make a rapid response speed, to achieve the continuous trading.

Man is the most important

Easy speed Lutenike Chairman and CEO Howard (Howard Lutnick) appears on television to announce his company suffered a devastating blow. When he talked about the passing of so many lives, and the company will be faced with the challenge of rebuilding, he cried. He vowed will do our best to rebuild his company and employees to ensure the families of the victims will bear the responsibility.

Lutenike launched the emergency plan, while employees of personnel responsible for determining the casualties, and to appease the families of the victims of emergencies such as the work of the staff, other organizations for the restoration of the surviving elite operators do all possible efforts.

Easy to speed through the company's Web site published a list of missing persons, at any time announced by various means of accurate information. Easy-speed lost a total of 180 employees, including half of its U.S. headquarters technical staff. The company's technical staff lack the elite speed easily, such as: easy to speed President and COO Frederick T 路 Valazzi (Frederick T Varacchi), total disaster emergency recovery plan architect 绾︾憻澶悏濂?Koni ( Joseph Giaccone).

However, easy to speed the company still has some significant wealth survived, most of the senior technical director of personnel was not in the company, including the company's current chief technology officer easy to speed, but also Noviello's right-hand man 椹壒鍏嬪姵鏂?( Matt Claus), he is also a fishing trip was scheduled to participate in the activities.

Noviello said that this story has a lot of luck. A senior network engineer survived being late for work, he was late because his wife baked for her daughter to school to eat the cake, had to go to grocery store to buy all the necessary ingredients. Some employees are vacation time, and some officers are simply just not been offered in three shifts 24 hours a day in the schedule. In the end, the statistics are 306 disaster survivors, including 260 information technology staff.

306 survivors turn grief into strength is operating resume fighting threw himself into the best explanation. Soon, Noviello, NJ Rochelle district rush back to work organization, where a company trading systems mirror site. The next day, some staff were immediately sent to Rochelle area, operation and management of all can not use the system.

Lutenike told Noviello, the bond market will be on September 13 re-opened the morning and asked him whether IT systems are ready. Noviello sure that everything was all right. He said: "We never thought at the time not to get everything ready, our team is full of passion and dedication. We tell ourselves, for our friends and our own, everything would be ready."

Noviello most of the next two days will be divided into two parts, one in Manhattan borrowed makeshift command center, the other one is at home, use the phone and landline contact with the outside world. Even so, he will have frequent communication system to connect the company's London, has been unable to get the various data obtained from New York.

Each two-hour conference call on important events and keep tracking the target. Noviello said: "We have been two o'clock when the discussion of 4:00, they also have been discussing, in this time who will sleep? Work is the best treatment." Noviello and his team in the cold and crowded data center work, relying on their understanding of the systems and procedures, rather than according to a written plan to work. Senior staff were on shift work, regular employees in accordance with the need to work. As small companies, easy to speed a large number of employees the opportunity to face different systems.

Be prepared for

Despite the enthusiasm high, Noviello aware of the difficulties they face are many. Noviello said: "We have a number of important problems to overcome, we have a systematic approach and begin to solve the Zhexie problems, Mei Mei You do step and should be the work of another scheme for. Those days were very chaotic, Dan Shi employees who know they need to do something. "

Need urgent problems are: to restore customer private network and the company's trading system easy to speed connection, replace processed through the banking system and cancel the deal to restore the World Trade Center office in lost data, survived for the 306 employees to find workplace and work equipment accordingly.

Help from other companies, became easy to speed critical for rapid recovery. ADP, the system makes the back part of the maintenance work all night, Wednesday 2:00 Compaq transported to a hundred computers, Cisco quickly provide Internet Protocol-based telephone system. The Microsoft NT system maintenance is sent to the trade group to help speed the company, because then easy to speed their own servers and desktop maintenance team has suffered as well. Meanwhile, Pratt & Whitney UBS (UBS PaineWebber) in Manhattan easily speed provided a temporary office. When used for file backup tape drive is insufficient, Klaus 11 o'clock at night drove to another company, to borrow a friend.

Of course, everything can be smooth or as easy to prepare the appropriate rate plan, to organize resources, and thus can not accept the failure of various causes of disruption to make quick response. Carried out after some thought, Noviello that this series are based on the earlier preparation in disaster recovery, temporary work and not technical. Had no backup site in Rochelle Zone, even to make another big effort to get more help and will not be effective. Noviello said the sentence is confirmed be prepared for an old saying. Do not buy anything, should be one, but should at least buy two, the fact that this is a wise habit.

As a first step towards recovery, Noviello to the customer's communication situation and the impact on the actual transactions made an accurate judgments. He had no way contacted by phone or pager colleagues in the building, so he called the company to easily speed offices in London, at the World Trade Center to understand whether the system can work normally.

Quick and easy to speed the company's system in New Jersey, Rochelle area to build up each machine, all kinds of connections and features make the copy, through third-party facilities in London, the World Trade Center is located in temporary offices and Rochelle area office locations connected.

From the first day, Rochelle area, where was seen as a collaborative office space, disaster recovery, not a temporary office. This change is because the speed is easily the largest electronic bond trading participant must provide uninterrupted service. Rochelle area in the mainly blue-collar towns, no feature of this building is ideal for office, especially a telecommunications building on the opposite side to ensure a reliable communication.

Mirror sites and the World Trade Center, where high-speed optical fiber connections, through this connection, easy to speed the company's storage area of each local network connections together. Data replication software can do around the point of critical database mirroring processing. World Trade Center in the main data center and the mirror sites to switch between each other, special product information stored in one place for one month and then switch to another place, about 40 products, all transaction information in two places at any time are available. Noviello said: "can do that, thanks to the recent disaster recovery test we have done."

Two Problems

Noviello was soon clear to resume business operations to stop two huge obstacles: customer-specific network connection can not mirror system, and banks can not connect cause can not process transactions.

Easy to speed critical business broker transaction processing system are mirrored in the Rochelle area, intact saved. However, all customers of the private networks are connected by fiber to the World Trade Center, and now no one can easily speed private network access server in Rochelle area to perform transactions, to restore connectivity becomes paramount.

September 11, Yi-speed service agent companies to the private network customer base is still in the World Trade Center. At that time, easy for high-speed connection to the network to the public telephone network in the Rochelle area plan has not been determined. Noviello said: "We started construction in February, it was not yet completely ready, there are some places we want to make a final finishing."

No way to connect public telephone network, but the Rochelle area site can connect to the Internet, you can also connect to the data center in London. So Noviello team immediately contact customers to arrange other ways to easily connect to the company's private network speed.

For overseas customer proprietary network, easy speed change their cable network to London. Easy to work with speed and customers to re-configure the server to connect to London to prepare for, and change or expand the customer to connect to the Easy-speed network permissions. For there is no dedicated network of overseas customers, easy to speed so that they connect to the Internet, and enable customers to connect to the Rochelle area system. Verizon Communications has helped accelerate the installation of voice lines, and accelerate the transmission speed digital lines.

However, to solve this problem is to process transactions more difficult. Although Noviello team to restore some applications to run back-office functions of financial transaction systems, but not as easy to speed all the products in the Rochelle area site to rebuild a processing system. Unable to handle and cancel operations, easy to speed the company can not resume operation.

Here, speed was easily rival ICI / ADP for help. ICI / ADP is also an electronic trading company, which through their own connections with the banks to process transactions to help trade-speed solve the problem. On Wednesday night, easy speed to the team with the detailed planning of the system's own background, ADP system, and successfully test transaction data sent to Morgan and other banks.

Easy to speed the company's stock from October first week of re. Its share price by the September 10 when to stop trading down 8.69 U.S. dollars to close on October 5 of 5.91 U.S. dollars. The loss of so many employees and closed the voice broker services business, the company's strength was weakened. However, the company, after all, survived.

As the current disaster recovery plan that allow companies to continue operating. After that, the system easy to speed the company's ability to fight disasters more, even if the accident occurred Rochelle area, but the company's data center in London, as the mirror sites can still continue to provide services. Noviello said: "We are drawing a lot of lessons to rebuild the system, including how to make the system able to support more automation, we will not before the bad habits bring back, but will train more good habit. "

8:30

The pleasant planned fishing trip, imagine the fun possible.

8:47: World Trade Center North Tower hit by plane, could affect the data center.

8:50: received the company's systems group manager Jimukefei the phone, he was rushed to the area in New Jersey, Rochelle way mirror sites.

8:51: In the World Trade Center, 103 level green data center, communications with London is still normal.

9:03: Another plane hit the South Tower was.

9:50: South Tower collapse occurred.

10:29: North Tower collapse, and London offices and data centers, communication interruption.

11:00: Information technology-member emergency team was sent to Rochelle area to determine the situation there.

12:00: The first turning point in the restoration of services there, and technicians continue to search for the missing.

Press Notes

Disappeared in the bank

In three days time from 9.11 to 9.13, easily released a four-speed news release. In every one manuscript, the easy to notice the speed of the latest company to express the company's determination to resume operations. At the same time, using all possible means publication of a crisis, contact telephone number, such as: media contact number, contact telephone number so staff relatives.

The contrary, the country's banks disappeared place in their hearts, especially in times of emergency. Recently, a state-owned large banks because of online banking security, to be class actions. In the bank website, do not see any news of the lawsuit, could not find the media contact person's telephone. Of course, that is, usually, a large state-owned bank in any website, even with the international practice is known as the listed banks, only to find customer service phone.

Incidence of fraud, one after another of the internet banking disputes, difficult to resolve non-performing loans, stock, etc. at any time possible to schedule any bank in China pushed to cusp. Just to the Chinese banking market, but also a lack of experience and media contacts, not to mention the crisis in public relations, and disappeared, Bierbujian seems to be tacit choice of bank chiefs.

However, there is not an unventilated wall disappeared and can not prevent the introduction of negative reports, only for all kinds of speculation, gossip spread faster. It appears that not only charges with international standards, as well as crisis management, public relations.

At 8:00 am on September 13, 911 47 hours after the incident, easy to restore the running speed.






相关链接:



States UNITED States to transfer?



E-cology in the Pan Micro Series 14



Extreme PROGRAMMING (eXtreme Programming) Summary



Error Accessing And Deleting Messages In Pst After



Articles about Communications Tools



Web Servers Comparison



introduction Games SIMULATION



FLV to 3GP



DVR-MS to MP4



E-cology In The Pan Of A Micro-series



Why "Chinese made" did not engage in head



Qihoo 360 Xiemoshalv download Station block group



NOKIA free navigation overtly



China Unicom Confirmed That IPhone Talks Progress