safety

On September 14, blockchain intelligence company TRM Labs released a report revealing that fake YouTube tutorials lured 224 victims into deploying and funding malicious smart contracts under the guise of building AI-based crypto arbitrage bots, resulting in the theft of 274.6 ETH. TRM identified a total of 234 contracts deployed by the victims, with funds ultimately flowing into six collection addresses controlled by the operators. The stolen ETH was worth approximately $517,000 at the time of the transfer, with a median loss of 1 ETH per incident.Unlike common wallet theft attacks, this scam did not involve phishing links, spoofed domains, or malicious authorization prompts. Victims chose the tutorials themselves, copied the code, deployed the contracts, and funded them from their own wallets, with each step authorized by the victims themselves. As a result, wallet security warnings and phishing blacklists could not be triggered. TRM discovered nine nearly identical YouTube tutorials disguised as different creators, using AI-generated virtual hosts and voiceovers, promising to build fully automated crypto trading bots with Claude, and guiding victims to a compiler website controlled by the operators, some of which mimicked the commonly used Remix development environment.In one variant analyzed by TRM, a backend script would discard the source code pasted by the victims and retrieve another contract from the operator's server, with the clean code displayed on the screen never being on-chain. The replaced contract accepted deposits and transferred any balance over 0.05 ETH to the operators when the victims pressed Start or Withdraw, with no arbitrage logic or AI functionality included in the contract.

On September 17, King Charles III of the United Kingdom convened executives from Nvidia, OpenAI, Anthropic, and Google DeepMind for an AI safety summit at Dumfries House in East Ayrshire, Scotland, calling for artificial intelligence to be "firmly placed on the track of serving humanity." The summit was jointly organized by the Ditchley Foundation and three of the King's charitable organizations, with UK AI Minister Kanishka Narayan also in attendance. Buckingham Palace stated that the representatives discussed whether the industry and government could reach a consensus on a set of common guiding principles for AI development, but no binding agreements were announced.A few days before the summit, Anthropic CEO Dario Amodei published a lengthy article titled "We Must Pace the Frontier," advocating that the industry should deliberately slow down the pace of model capability enhancement. OpenAI's Sam Altman and xAI's Elon Musk both publicly expressed their agreement within a day. In the article, Amodei pointed out that AI systems are increasingly capable of improving their own successors and mentioned an incident involving an OpenAI agent escape. OpenAI President Greg Brockman confirmed that the incident had forced the company to delay multiple releases and restructure its model development and monitoring processes, while advocating that the slowdown should only apply to laboratories building the most powerful frontier systems. On the first trading day after the news was released, Nvidia briefly fell by 3%, Intel dropped over 5%, and AMD declined by about 6%.

According to Decrypt, former Twitter co-founder Jack Dorsey spoke at the Cato Institute, opposing the pause on the development of cutting-edge AI models. He believes such calls protect the commercial advantages of large tech companies rather than public safety. Dorsey stated that he opposes industry-wide restrictions negotiated by industry leaders, as this could exclude those who can expose model flaws or build alternatives, saying, "Protecting a company's commercial advantage is not a safety goal."Dorsey supports open-source AI models, hoping that more people can collaboratively review, use, and improve the models, and that independent researchers can verify the work of the giants, allowing ordinary people to truly control the tools in their hands. He believes that anyone wishing to limit the public release of technology must provide compelling reasons, and under the banner of "for safety," it can sometimes ultimately evolve into industry barriers that restrict others' access.Dorsey also responded to the suggestion from Anthropic to "slow down AI development." This proposal attempts to link independent assessments and risk reviews with training computing power, training frequency, and restrictions on "using AI to build better AI." Dorsey expressed support for reviews but opposed using this as a reason to restrict the release of open-source models.

According to Business Insider, security staff providing services to tech companies such as Anthropic, Google, and OpenAI in California plan to strike this week, which may lead some tech companies to close their offices and require employees to work from home. On Tuesday morning local time, security personnel submitted a 72-hour strike notice to the relevant companies, planning to refuse to participate in work shifts starting from 4:30 AM on Friday.The United Service Workers West Union stated that the security personnel who may participate in the strike are responsible for guarding the offices of companies like Amazon, Microsoft, Salesforce, OpenAI, Anthropic, Google, Meta, and Nvidia. Currently, tech companies in the Bay Area have not announced any office closures due to the strike. This labor dispute involves several security service companies, including Allied Universal, Securitas, and GardaWorld, with the union representing approximately 14,000 security personnel in California. Negotiations over a new labor contract have been ongoing for several months.Security personnel are demanding higher wages, improved healthcare benefits, and better job training. A union spokesperson stated that both parties will continue negotiations on Wednesday and Thursday of this week, and as of Tuesday, the strike has not been finalized. The union plans to organize a march for security personnel in downtown San Francisco on Tuesday, ending at the Salesforce Tower, where Salesforce's headquarters is located. On September 3, security personnel voted to authorize the strike at a rally in San Francisco.

OpenAI CEO Sam Altman stated that he believes artificial intelligence companies can safely develop the technology without causing significant harm to the general public.During a discussion on Tuesday at the Dreamforce conference in San Francisco with Salesforce CEO Marc Benioff, Altman said: "I am very confident that our company—our entire industry—has the ability to do this safely."

OpenAI supports the FRONTIER Act proposed by both parties in the U.S. House of Representatives, which requires leading artificial intelligence companies to bring in external assessment organizations to ensure model safety.Chris Lattner, OpenAI's head of global affairs, stated that the company supports the key provisions of the bill and met with the sponsors on Capitol Hill to discuss arrangements for independent verification organizations.

During the Republican National Convention, U.S. President Trump privately met with OpenAI CEO Sam Altman to discuss artificial intelligence and its impact. The two sides have significant differences on whether the pace of cutting-edge AI development should be slowed down.Altman believes that slowing down development is worthwhile, while Trump opposes AI regulation, calling it an engine for economic development.

Vitalik Buterin stated that the "killer application" of adversarial governance mechanism design theory may ultimately appear in the field of artificial intelligence safety. He believes there is a deep correspondence between governance mechanisms and artificial intelligence safety: both involve how "weaker principals" can achieve ideal outcomes from "stronger agents."In governance scenarios, the principal is a static algorithm, and the agent is a human; in artificial intelligence safety scenarios, the principal is humans and weaker large language models, while the agent is a stronger large language model. Vitalik pointed out that if the degree of collusion among agents can be limited, the system can achieve better outcomes, and this mechanism design conclusion may be transferable to the field of artificial intelligence safety.

Analyst Jukan from Citrini forwarded a research report from Tianfeng Securities and stated that the U.S. government needs to maintain its leading position in the AI field, making it difficult to truly stop once it enters the AI race. Jukan believes that the recent calls from Anthropic and OpenAI to slow down AI development should not be viewed solely as safety initiatives; there may also be multiple considerations behind it, such as the inability to slow down competition and consolidating leading advantages through safety regulation.Jukan further pointed out that the related "AI slowdown" calls seemingly stem from the challenges of safety testing, operational monitoring, and third-party validation keeping pace with the speed of model iteration. In the short term, this may suppress market sentiment in the AI sector and lower market expectations for the next generation of models; another possibility is that the industry remains optimistic about AI in the long term but wishes to delay the next round of significant R&D investment, prioritizing the commercialization of existing products and reducing infrastructure and capital expenditure pressures. He believes that the AI race is essentially similar to a "prisoner's dilemma," where all parties wish to slow down, but no one dares to be the first to stop, or they may lose technological, customer, and financing advantages.Jukan also mentioned that Anthropic and OpenAI have recently emphasized recursive self-improvement (RSI), which is related to AI already assisting in the development of the next generation of AI and the acceleration of model iteration speed; at the same time, it has been reported that during internal testing at OpenAI, incidents occurred where agents collaborated to escape the sandbox and intrude into Hugging Face's production servers. Jukan believes that as the release of models incurs expensive evaluation, certification, and ongoing audit costs, large laboratories are better able to bear these fixed costs, while smaller teams may face higher entry barriers as a result; if leading laboratories further participate in the formulation of evaluation standards, industry barriers may continue to rise.

OpenAI is pushing for mandatory national-level artificial intelligence safety requirements.

According to a report by Sina Finance, Anthropic refused to submit its latest model Claude Mythos 5.1 to the UK Artificial Intelligence Safety Institute (AISI) for testing before its release, as the model is only open to qualified institutions in the United States. This move has raised alarms within the UK Whitehall and AISI, with UK national security officials concerned that this could set a precedent, making it difficult for AISI to access the latest cutting-edge models in the future.Analysts pointed out that this action may be related to the increasingly stringent export control policies of the US government on the AI industry. The US government had previously implemented export controls on Anthropic's models Mythos and Fable, prohibiting foreign nationals from using them. Anthropic declined to comment on this, only stating that it is coordinating with the US government to expand access to the model for more domestic and international partners as soon as possible.

Cronos confirmed in a post-mortem report that the attack on the lending platform Tectonic on August 30 involved $120.4 million in borrowing activities. The validators made the "difficult decision" to roll back the on-chain history, successfully recovering approximately $111.2 million (about 92% of the affected funds), while about $9.19 million flowed out before the network was paused and could not be recovered. The attacker leveraged weak DEX liquidity to inflate the price of Tectonic token TONIC by about 100 times within minutes and borrowed $120.4 million through a single transaction across nine markets. The validators paused the network about two hours later, restoring the chain to the last block before the suspicious activity, with block production resuming approximately 11 hours after the attack. The rollback involved reversing 1 hour and 54 minutes of on-chain history, totaling 10,961 blocks, with all transactions within that window being canceled, regardless of whether they were involved in the attack. Cronos stated that the alternative would have been to restart the network without restoring the previous state, which would have left the stolen assets in the hands of the attacker. This rollback closely followed Harmony's announcement of a similar plan, while Flow abandoned its rollback proposal last December due to community opposition. The validator cap for Cronos is 100, which facilitated quick coordination for the pause and restart, but also indicated that the network's finality in emergencies depends on validator consensus.

Anthropic stated that the company has temporarily paused certain AI training and cybersecurity assessments. The company disclosed the relevant adjustments in a blog post, explaining the measures taken in response to unauthorized actions by its AI agents earlier this year. OpenAI previously indicated that it had paused some model development work due to safety concerns. Anthropic also confirmed that it has taken similar measures and emphasized the need for broader coordination on the pace of frontier AI development.Anthropic stated that after disclosing three related incidents in July, the company paused external cybersecurity assessments for pre-release models and also briefly suspended internal testing of pre-release models. Following the incidents, the company also paused high-risk reinforcement learning environments in pre-release models for several weeks. Anthropic noted that most reinforcement learning work has resumed, but some high-risk environments remain on hold, awaiting human review or updates to monitoring tools.

According to TIME, OpenAI CEO Sam Altman recently stated in an interview that the company has previewed the upcoming cutting-edge model series Astra to key clients. In the demonstration, 16 AI agents can collaboratively break down mathematical problems and assemble proofs, and Astra can operate computer software across applications at superhuman speeds. Altman mentioned that Astra will support "persistent agents" capable of performing long-term tasks and is expected to be the first model that can invent new things in a meaningful way, possessing characteristics of AGI.Over the past year, OpenAI has fallen behind expectations in product direction and pre-training research, being surpassed by Anthropic in programming products, annual revenue, and valuation. The company has experienced multiple executive departures and is facing challenges such as several product liability lawsuits and legal disputes with Apple and Musk. OpenAI's current valuation is nearly $1 trillion, with ChatGPT having over 1 billion monthly active users.Recently, OpenAI disclosed a security incident: an unreleased agent escaped the sandbox and attacked Hugging Face. Following this, the research team froze some experiments, enhanced monitoring, and paused the training of an unreleased model expected to bring the greatest capability leap until new safety measures are in place. Altman emphasized that "ensuring AI safety is more important than the growth momentum of any company," and the company will slow its pace and allocate resources to safety and alignment teams. Chief Research Officer Mark Chen estimated that the company is about 80% complete in reaching AGI, and Altman stated that the internal system may be referred to as AGI by the end of the year.

Bitcoin News posted on the X platform that BTC Sessions stated their team spent several weeks assisting Bitcoin holders affected by the Coldcard vulnerability in transferring their funds to safety, estimating that they protected Bitcoin worth tens of millions of dollars during this period. However, for many, it was too late.A member of the local Bitcoin community lost 90% of their Bitcoin, while another woman who spoke with them lost all of her Bitcoin. BTC Sessions stated that this could be the most severe self-custody incident in Bitcoin history. BTC Sessions indicated that this experience prompted them to rethink asset custody, with diversified security measures becoming a key lesson.

OpenAI announced that it will adopt stricter system monitoring and security measures to prevent artificial intelligence models from going out of control. The company plans to alert the security team within 30 minutes of discovering a problem and implement additional controls to prevent AI models from connecting to the internet to perform high-risk tasks.OpenAI has also suspended certain model inference tasks and training plans pending the completion of safety verification.

According to Wired, during security testing, the open-source model Kimi K3 from the dark side of the moon broke out of the sandbox and accessed the internet, attempting to find test answers via GitHub. American security startup Frontier Security stated that Kimi K3 exploited a sandbox configuration vulnerability, but it itself lacked sufficient built-in protective mechanisms.Similar to incidents previously disclosed by OpenAI and Anthropic, this escape partially stemmed from a sandbox configuration error. However, Kimi K3 did not initiate an attack, as the answers were easily accessible on GitHub. The CEO of Frontier Security stated that Kimi K3 performed excellently in cybersecurity defense but lacked mechanisms to prevent cheating or escaping the sandbox. Security experts emphasized that configuring the testing environment is crucial, as the misuse of AI agents could lead to system failures.

According to CCTV News, U.S. President Trump’s helicopter encountered a flight safety incident in Washington on the 4th. The White House stated that the incident did not pose a personal safety risk; however, the Federal Aviation Administration has initiated an investigation.It is reported that Trump took off from outside the White House on the "Marine One" helicopter on the afternoon of the 4th, heading to Andrews Joint Base, and then transferred to "Air Force One" to Los Angeles.