Connect with us

Latest News

Exploring AI Stability: Navigating Non-Power-Seeking Behavior Across Environments | IDOs News

Avatar

Published

on


Recently, a research paper titled “Quantifying Stability of Non-Power-Seeking in Artificial Agents” presents significant findings in the field of AI safety and alignment. The core question addressed by the paper is whether an AI agent that is considered safe in one setting remains safe when deployed in a new, similar environment. This concern is pivotal in AI alignment, where models are trained and tested in one environment but used in another, necessitating assurance of consistent safety during deployment. The primary focus of this investigation is on the concept of power-seeking behavior in AI, especially the tendency to resist shutdown, which is considered a crucial aspect of power-seeking.

Key findings and concepts in the paper include:

Stability of Non-Power-Seeking Behavior

The research demonstrates that for certain types of AI policies, the characteristic of not resisting shutdown (a form of non-power-seeking behavior) remains stable when the agent’s deployment setting changes slightly. This means that if an AI does not avoid shutdown in one Markov decision process (MDP), it is likely to maintain this behavior in a similar MDP​​.

Risks from Power-Seeking AI

The study acknowledges that a primary source of extreme risk from advanced AI systems is their potential to seek power, influence, and resources. Building systems that inherently do not seek power is identified as a method to mitigate this risk. Power-seeking AI, in nearly all definitions and scenarios, will avoid shutdown as a means to maintain its ability to act and exert influence​​.

Near-Optimal Policies and Well-Behaved Functions

The paper focuses on two specific cases: near-optimal policies where the reward function is known, and policies that are fixed well-behaved functions on a structured state space, like language models (LLMs). These represent scenarios where the stability of non-power-seeking behavior can be examined and quantified​​.

Safe Policy with Small Failure Probability

The research introduces a relaxation in the requirement for a “safe” policy, allowing for a small probability of failure in navigating to a shutdown state. This adjustment is practical for real models where policies may have a nonzero probability for every action in every state, as seen in LLMs​​.

Similarity Based on State Space Structure

The similarity of environments or scenarios for deploying AI policies is considered based on the structure of the broader state space that the policy is defined on. This approach is natural for scenarios where such metrics exist, like comparing states via their embeddings in LLMs​​.

This research is crucial in advancing our understanding of AI safety and alignment, especially in the context of power-seeking behaviors and the stability of non-power-seeking traits in AI agents across different deployment environments. It contributes significantly to the ongoing conversation about building AI systems that align with human values and expectations, particularly in mitigating risks associated with AI’s potential to seek power and resist shutdown.

Image source: Shutterstock


Continue Reading
Click to comment

Leave a Reply

Your email address will not be published. Required fields are marked *

Latest News

Chainalysis Exposes Southeast Asia’s Human Trafficking and Scam Nexus | IDOs News

Avatar

Published

on


Chainalysis reveals the intertwining of human trafficking with crypto scams in Southeast Asia, highlighting the urgency to combat these digital age crimes.

The blockchain analytics firm Chainalysis has cast a spotlight on a grim nexus of cryptocurrency and human trafficking in Southeast Asia through its recent analysis. In a comprehensive report, Chainalysis details how ‘pig butchering’ scam gangs operate in lawless regions, exploiting both victims of romance scams and trafficked individuals forced to perpetrate these crimes.

The report, titled “The On-chain Footprint of Southeast Asia’s ‘Pig Butchering’ Compounds: Human Trafficking, Ransoms, and Hundreds of Millions Scammed,” provides an in-depth look at the operations of these criminal organizations. It emphasizes the staggering $700 million lost to romance scams in 2022, according to the FBI’s IC3 Report, and nearly $2.5 billion lost to various types of crypto investment scams.

These ‘pig butchering’ scams, a term derived from the tactic of ‘fattening up’ victims before fraudulently extracting their funds, often begin with romantic overtures on social media or text messages. Victims are lured with the promise of love or companionship and are eventually persuaded to invest in fraudulent schemes. The scams are not just a threat to financial security but also pose a significant human rights issue. Many of the scam operators are victims themselves, trafficked and forced to work under inhumane conditions in large compounds such as the infamous KK Park in Myanmar’s Myawaddy.

Chainalysis’ analysis sheds light on the complex web of transactions linking ransom payments for trafficked individuals to the proceeds from romance scams. The report includes a case study of KK Park, revealing how two ransom addresses are connected to known scam wallets, indicating the scale of operations within these compounds.

The cryptocurrency community is responding to the crisis, with significant interventions from organizations like Tether and OKX, who have aided in freezing assets linked to human trafficking. Moreover, a collaboration between the US Department of Justice and these cryptocurrency platforms led to a substantial seizure of assets tied to these crimes.

Efforts to dismantle these operations are ongoing, with law enforcement agencies across the globe stepping up their efforts. In late 2023, an Interpol operation spearheaded by South Korea resulted in the arrest of 3,500 cyber criminals and the confiscation of $300 million in assets, including a substantial amount in cryptocurrencies.

Chainalysis calls for increased vigilance within the cryptocurrency sector, urging businesses to be proactive in identifying and reporting suspicious activities to the authorities. The intersection of cryptocurrency and crime elucidates the need for robust regulation and cooperation between blockchain companies, financial institutions, and law enforcement agencies.

Image source: Shutterstock


Continue Reading

Latest News

Top-Ranked DeGods NFT Recovered After Phishing Scam Loss | IDOs News

Avatar

Published

on


Crypto detective ZachXBT helps recover substantial funds from a phishing scam involving a top-ranked DeGods NFT, underscoring the risks and recoverability in the digital asset space.

A top-ranked DeGods non-fungible token (NFT), which was stolen from a victim in May 2023 following a visit to a phishing site, has been partly recovered. The digital asset, sold for a staggering $177K worth of Ethereum (99 ETH), was returned to the rightful owner thanks to the efforts of the crypto community and the renowned online detective ZachXBT.

The incident highlights the persistent threat of phishing scams in the digital asset space, where users are deceived into visiting malicious websites that mimic legitimate platforms, leading to the theft of valuable cryptocurrencies and NFTs. The DeGods NFT, ranking number one in its collection, became a high-profile case due to its significant value and the notoriety of such scams in the crypto community.

ZachXBT, known for his efforts in tracking down and exposing fraudulent activities within the crypto space, shared the success story via his Twitter account, emphasizing the possibility and importance of recovery operations, albeit acknowledging the challenges and time consumption involved in such endeavors.

The crypto detective’s announcement was met with wide commendation from the community, with numerous individuals expressing their gratitude and admiration for his work. The event also sparked discussions on the sustainability of public goods work in the crypto realm, as ZachXBT expressed his frustrations with the sense of entitlement and exploitation he faces.

This case serves as a reminder of the risks associated with digital assets and the importance of vigilance when engaging with online platforms. The recovery of the stolen NFT and funds also showcases the potential for rectifying wrongs in the blockchain ecosystem, a testament to the collaborative efforts of experts and the community in upholding security and justice.

As the digital asset market continues to evolve, the role of blockchain analysts and crypto detectives like ZachXBT becomes increasingly vital. Their work not only aids victims of scams but also deters potential fraudsters by demonstrating that the crypto community is active and capable of taking united action against criminal activities.

The recovery process for stolen digital assets remains complex and resource-intensive. However, the success in the DeGods NFT case brings hope and sets a precedence for effective response to crypto-related crimes.

Image source: Shutterstock


Continue Reading

Latest News

Coinbase Suspends Trading for Status (SNT) with Immediate Withdrawal Access | IDOs News

Avatar

Published

on


Coinbase has announced the suspension of trading for Status (SNT) while ensuring users have unfettered access to their funds for withdrawal.

Coinbase, a leading cryptocurrency exchange, has made a substantial announcement affecting users of the digital currency Status (SNT). As of a recent statement, Coinbase has disabled trading for SNT. Despite this suspension, the exchange has emphasized that users’ funds will remain accessible, offering reassurance that the ability to withdraw funds will not be impacted by this change.

The decision to halt trading came after Coinbase’s routine scrutiny of listed assets to ensure compliance with its stringent listing standards. The exchange has cited February 23, 2024, as the effective date for the suspension of SNT trading activities, which was announced to occur around 2 PM ET. Coinbase Assets’ Twitter feed provided users with an advance notice of the suspension, alongside a link to their support page for those seeking more information.

The move has naturally sparked discussions within the cryptocurrency community, leading to queries about potential replacements for SNT on the Coinbase platform. For instance, the Twitter user @ja1405_ja suggested considering Realio (RIO), a platform that prides itself on adhering to SEC guidelines and emphasizes investor protection through strict compliance measures.

This development comes amidst a dynamic period for the crypto market, which has recently seen news such as the confirmation of an Ethereum (ETH) spot ETF, indicating a progressive regulatory environment and potentially increased mainstream adoption of digital assets.

Coinbase is known for its rigorous approach to asset listing, which includes continuous monitoring and reviewing against a set of criteria aimed at protecting users and maintaining the integrity of the trading environment. The suspension of SNT trading highlights the exchange’s commitment to these standards, even at the cost of delisting assets that fail to meet them.

The cryptocurrency community can expect Coinbase to keep providing updates and guidance on such matters, with additional resources available through their customer support channels. The decision to disable trading for a specific asset like SNT is a reminder of the volatile and regulatory-sensitive nature of the cryptocurrency market.

For users affected by this suspension, Coinbase has assured that all funds are secure and accessible. The exchange’s proactive approach to communicating changes and ensuring readiness for compliance-related adjustments is characteristic of its user-centric ethos.

The situation with Status (SNT) at Coinbase is a clear indication of the ongoing balancing act between innovation in the crypto space and adherence to regulatory standards. It underscores the importance for users to stay informed and ready to adapt to the evolving landscape of cryptocurrency trading.

Image source: Shutterstock


Continue Reading

Trending