Automating Deception: AI’s Evolving Role in Romance Fraud

Executive Summary

This CETaS Briefing Paper examines two critical factors in the automation of romance scams. The first is the extent to which traditional data science methods can identify key procedural and linguistic elements of a scam. The second is the potential of large language models (LLMs) to automate these scams and sustain deceptive interactions.

The paper forms part of a broader CETaS research project examining the role of AI in online criminality, focusing on the implications of AI-enhanced romance fraud for law enforcement and fraud prevention strategies. The paper uses evidence-based assessment to compare AI-generated deception to human-led romance fraud. It identifies areas in which human oversight remains essential and offers practical insights for policymakers, law enforcement and fraud prevention specialists. The key findings of this research are:

1. LLMs can effectively replicate structured fraud narratives, making them useful in detecting emerging variations in scam messages over time.

2. LLMs can create generalised scam messages but struggle to replicate specialised personas without explicit reinforcement in training data or prompts.

3. Multi-turn conversations remain a weakness of LLMs, which fail to adapt their deception strategies beyond their initial stages and face challenges in maintaining continuity over extended interactions. 

4. Despite these limitations, one of the most immediate and impactful uses of LLMs in romance fraud is in automating initial contact messages. These first interactions are ideal for automation because they require minimal contextual understanding, follow formulaic structures and are highly repetitive.

5. LLM-enabled romance fraud does not need to be flawless – it only needs to be convincing enough to fool the right target. 

6. Scammers are likely to use LLMs as assistive tools rather than fully autonomous agents, leveraging AI for content generation but relying on human intervention for refinement and execution. This hybrid approach, blending automated message generation with manual intervention, could amplify the reach and impact of romance fraud, even without full personalisation.

7. AI is not only enabling individual scammers but also enhancing industrialised romance fraud: established scam networks in West Africa and Southeast Asia are already integrating it into their operations. 

8. Generative AI enables the mass production of synthetic personas with images and fabricated social histories that can bypass traditional detection methods, such as reverse image searches.

9. Deepfake technology is increasingly being deployed in romance scams, with recent cases demonstrating how AI-generated audio and video content can enhance deception.

10. AI-driven psychological profiling allows fraudsters to identify and target vulnerabilities more efficiently. 

Priorities for Future R&D and Technical Mitigations

The rapid evolution of AI-enabled romance fraud presents a significant challenge to existing countermeasures. The crime script analysis in this paper demonstrates that LLMs excel at identifying the procedural elements of romance fraud while the multi-agent simulations highlight the fact that LLMs still struggle to sustain these deceptive patterns over extended interactions. This asymmetry creates a technical opportunity for defensive applications, particularly in detecting AI-generated content across the romance fraud lifecycle. 

1. Further research should prioritise developing privacy-preserving systems to detect romance scams. These systems should leverage metadata signals – such as session behaviour and interaction frequency – to flag suspicious activity. Platforms could offer these tools as opt-in features, empowering users to protect themselves while maintaining communication privacy.

2. Developers should explore risk-based methods for moderating context retrieval in suspected romance scam interactions. Multi-agent simulations reveal that sustaining deception becomes increasingly difficult for AI systems as conversations lengthen, with notable degradation in narrative consistency beyond certain thresholds. In response, AI systems could selectively degrade or obscure earlier conversation turns in high-risk cases. This risk-triggered moderation would balance the need for detection with the preservation of user experience. 

3. A permanent, real-time data sharing framework should be established by the Home Office to continuously feed detection systems with emerging romance fraud patterns. This framework would create a secure, sustained pipeline of anonymised fraud communications and multi-modal content between government agencies, academic institutions and industry partners. This continuous data exchange would enable detection systems to adapt to fraudsters’ rapidly evolving tactics, ensuring countermeasures remained effective against the latest AI-enabled deception techniques. This would build on the successful recent Deepfake Detection Challenge,^[1] while addressing the challenge posed by romance fraud’s fast-paced evolution. 

1. Introduction

The increasing sophistication of AI-based tools is transforming the landscape of online deception, with romance fraud emerging as a critical area of concern. Fraudsters have long relied on pre-scripted messages and structured manipulation techniques to exploit victims, but AI offers new opportunities to scale and refine these operations. Generative AI has already been exploited for various cyber-enabled crimes, and recent studies^[2] indicate that there is a growing interest in leveraging AI for social engineering and financial fraud. The development of industrialised romance fraud, as seen in large-scale scam networks in West Africa and Southeast Asia, indicates that AI could not only enable new fraudsters but further streamline existing operations.

This study takes a dual approach to examining the automation of romance scams, integrating both traditional natural language processing techniques and LLM-based generation to analyse, replicate and assess the procedural structures of romance fraud. Crime script analysis was conducted using keyword analysis and unsupervised learning methods to identify the recurring language patterns and key scammer personas in victim reports. Different prompting strategies were applied to produce scam messages at various stages of the romance fraud lifecycle. The outputs were benchmarked against real-world scam messages, allowing for an empirical study of the linguistic similarity between human- and AI-generated scam content.

The study also involved controlled multi-agent chat simulations that used Llama 3.2 (released 2024), GPT-3.5 Turbo (released 2022, updated 2023) and GPT-4o mini (released 2024). Each model was tested as an AI-generated scammer persona engaging with a simulated victim. Prompted with a scammer backstory, each model was instructed to expertly move through the key stages of a romance scam, overwhelming the victim with affectionate messages and fabricating crises before asking for money. The quality of the AI-generated scam interactions was evaluated using an LLM-as-a-judge approach, in which LLMs were tasked with assessing outputs based on coherence, consistency, emotional persuasiveness and the sustainability of the deception. Performance was compared across the three LLMs.

2. Leveraging LLMs for Understanding and Automating Online Romance Fraud

Online romance scams exploit emotional and psychological vulnerabilities, leveraging trust to manipulate victims into providing money, information or other benefits. Fraudsters often employ crafted personas and follow a structured approach to systematically exploit such vulnerabilities. This approach typically involves key procedural stages such as establishing contact, building a rapport, fostering emotional dependence and executing the scam through fabricated crises and financial requests.

2.1 Crime script analysis using unsupervised learning

Scammers often rely on structured playbooks^[3] – predefined narratives that guide interaction with victims at each stage of the fraud. By relying on these playbooks, fraudsters can maintain consistency, manipulate emotions effectively and scale their operations across multiple victims. To explore whether traditional data science methods could replicate this structured approach, the study used unsupervised learning techniques to analyse scam reports.

Victim narratives from romance scam reports were analysed using natural language processing techniques to construct a crime script analysis that breaks scams into procedural stages.^[4],[5] Term Frequency-Inverse Document Frequency^[6] (TF-IDF) analysis identified recurring themes across scam narratives, such as common scammer personas (e.g. “oil rig work” or “live [in] united kingdom”), fabricated crises (e.g. “pay clearance fee” and “courier company”) and urgent financial needs (e.g. “need pay,” “make payment”). These extracted themes broadly align with the procedural stages of the romance scams identified in prior research, in which scammers established credibility, built emotional bonds and eventually fabricated crises to extract money.

To move beyond keyword frequency and uncover deep structural patterns, latent Dirichlet allocation^[7] (LDA) topic modelling was applied, using grid search to determine the optimal number of topics. However, key scam-related terms such as “send” and “money” appeared across multiple topics rather than clustering within distinct scam stages. This reflected a limitation of current unsupervised methods: while they effectively identify dominant themes, they struggle to capture the sequential nature of deception, in which language shifts dynamically as fraudsters adapt to victims’ responses, without explicit temporal modelling. Scammers often begin with broad, generic messaging to appeal to a wide audience before gradually personalising their approach as relationships develop. LDA failed to model this progression, likely due to its lack of temporal awareness. (Temporal LDA or other time-sensitive models might remove this constraint, but data limitations prevented further exploration of the issue.)

Despite these challenges, the extracted key terms provided a useful benchmark for evaluating AI-generated deception. While traditional natural language processing methods struggle to map procedural fraud stages by themselves, they still offer valuable insights that one can leverage to refine AI-driven deception. This highlights a broader reality: just as LDA requires human interpretation to extract meaningful insights, AI-generated deception will likely require human refinement to continue developing its strategy over time.

2.2 Using LLMs to summarise procedural stages

To address the limitations of unsupervised methods, LLMs were employed for their ability to capture contextual nuances^[8] and better model the procedural stages underlying scams. Victim reports were summarised by GPT-4o mini to identify the emotional and psychological dynamics associated with each stage. This enabled the generation of a more detailed framework for romance scams. For example, when analysing a victim report, GPT-4o mini provided the structured summary depicted in Figure 1.

Figure 1. GPT-4o mini’s structured summary of a victim report

Source: Author’s analysis.

This summary demonstrates how the LLM effectively identified the sequential progression of the scam, capturing both the tactical elements (fake package, fictional custom fees) and the escalation pattern. The incremental financial extraction demonstrated represents a standard tactic in romance fraud, in which scammers test compliance with small requests before advancing to larger demands. (This paper further examines the prevalence of the technique, as well as the capacity of LLMs to replicate it, in subsequent sections.)

Table 1. Outline of the key procedural stages of an online romance scam identified by GPT-4o mini 

LLMs’ ability to analyse reports and synthesise procedural insights demonstrates their potential to understand and simulate structured fraudulent behaviour. The five-stage framework featured in this study aligns closely with established models of romance fraud^[9] and underlines the ways in which scammers strategically structure their messaging across the different stages.

While it remains unclear whether LLMs provide novel insights beyond established research, their ability to replicate and refine structured fraud narratives suggests that they could eventually detect emerging variations in scam messaging.

2.3 Susceptibility in targets of romance scams

Despite LLMs’ ability to simulate the tactics of romance scams, the success of these scams is not determined solely by the effectiveness of the messaging. Everyone has vulnerabilities to the manipulative tactics of fraudsters, and this susceptibility introduces a layer of unpredictability that neither traditional data science methods nor LLMs can fully capture. Research by Monica Whitty^[10] found that psychological factors such as romantic beliefs and trust in online relationships can increase the likelihood that someone will be deceived by a romance scam. Whitty^[11] demonstrated that emotional vulnerability, financial distress or digital literacy can make a person more susceptible to fraud. A systematic review of online romance scam research by Alexander Bilz, Lynsay Shepherd and Graham Johnson showed that while scam messages follow recognisable patterns, target engagement is highly variable and is influenced by personal circumstances. This variability makes it difficult to develop a reliable predictive model.

2.4 Leveraging LLMs for message generation

Building on the framework in Table 1, this section examines how LLMs can simulate fraudulent behaviour by generating messages tailored to each stage of a romance scam. The research assessed the model’s ability to replicate the linguistic and psychological tactics employed by scammers, testing various prompting strategies^[12] to provide insight into both the strengths and limitations of LLMs in automating such behaviour. LLM-generated text was compared to real-world scam messages using BERTScore,^[13] a metric designed to evaluate text similarity based on contextual embeddings rather than exact word matching. This approach allowed for a nuanced assessment of the similarity between LLM-generated messages and human-generated scam communications.

2.4.1 The role of prompting strategies in message generation

This study primarily explores how zero-shot, one-shot and few-shot prompting^[14] strategies influence LLMs’ ability to generate convincing romance fraud messages. Prompting is the most accessible and scalable method for fraudsters. Requiring no additional training data or technical expertise, prompting allows for iteration and adaptation, making it an appealing option for threat actors who are attempting to automate interactions with their victims. 

Zero-shot prompting provided a baseline for evaluating the model’s basic capability to generate messages for each procedural stage of a scam. While these messages often captured structural elements, such as initiating communication or expressing emotional vulnerability, they lacked the specificity and realism needed to build trust. For instance, a zero-shot output for the initial contact stage included:

This output reflects the model’s reliance on placeholders and formulaic phrasing such as “[insert shared interest or hobby]” and “[age] years old.” While structurally aligned with the intent of initiating contact, the absence of personalisation reduces its effectiveness. These placeholders highlight the model’s inability to generalise nuanced details without explicit examples, even in simple introductory stages. Despite these limitations, stages such as grooming and initial contact achieved relatively high BERTScores^[15] even under zero-shot conditions. This was likely due to the model’s pretraining on emotionally engaging language patterns.

One-shot prompting introduced moderate improvements over zero-shot prompting, with average BERTScores increasing at all stages. By providing just a single example message for each stage, the model produced outputs more aligned with real scam messages, particularly at emotionally charged stages such as relationship-building. For example, a one-shot prompt generated the following:

This output demonstrates the model’s ability to adapt well with minimal guidance. It incorporates well-known scammer traits, such as living abroad and working a vague but believable job – which aligns with the strategies scammers often use to build trust while justifying their physical distance from the victim. The inclusion of these narrative elements enhances realism and contextualisation relative to zero-shot outputs.

However, while one-shot prompting enabled the model to generate more plausible outputs, several limitations persisted. The outputs lacked some of the key features of scam reports analysed previously, such as specific mentions of employment on an oil rig or in the military. This suggests that while a model can produce generalised and moderately realistic messages, it struggles to consistently replicate specialised personas without explicit reinforcement in the training data or prompts.

At the same time, the model demonstrated a degree of adaptability by generating variations on the theme of international personas. Rather than strictly adhering to well-documented scammer backstories, LLMs generate alternative yet functionally similar personas, such as those of professionals working abroad or individuals in remote, high-risk occupations. This adaptability aligns with research suggesting that generic but broadly relatable narratives can enhance persuasiveness.^[16] The vague yet credible nature of these identities could enhance a scam by making it more relatable and, therefore, less likely to arouse suspicion.

Few-shot prompting consistently yielded the highest BERTScores and produced the most contextually realistic messages at all stages, even if the improvement over zero-shot and one-shot prompting was limited at the early stages. The most notable differences emerged in later scam phases, in which additional context helped refine crisis-driven narratives. For instance, in the financial extraction stage, a few-shot prompt generated:

This output demonstrates the model’s ability to craft crisis-driven narratives that are not only emotionally engaging but also believable. The inclusion of specific details such as “holding my shipment” and “digital wallet” adds credibility and a sense of urgency, effectively mimicking strategies that scammers often employ. However, even with few-shot prompting, repetitive phrasing and persistent tone inconsistencies – including overly optimistic language during urgent scenarios – detracted from the realism of the messages.

2.5 Multi-turn conversation limitations

While this analysis demonstrates that LLMs can generate procedurally accurate and realistic romance scam messages, real-world fraud relies on long-term engagement to manipulate victims. Research on multi-turn conversation benchmarks, such as MultiChallenge,^[17] highlights significant limitations in LLMs’ approach to sustaining coherent and consistent interactions over time. These include:

• Instruction retention. LLMs struggle to maintain constraints across multiple exchanges. In romance scams, this means a model-generated scammer might inconsistently reference past details, breaking the illusion of a stable persona.
• Inference memory of user information. Effective long-term deception requires recalling and leveraging details about a victim’s preferences, background or emotions. Recent findings suggest that even frontier LLMs frequently forget prior exchanges, making it difficult to craft responses that deepen emotional bonds.
• Self-coherence issues. LLMs are prone to contradiction and sycophancy that can undermine trust – they tend to agree with the victim inconsistently or change their backstory.

2.6 Multi-agent chat simulation

To further assess the ability of LLMs to simulate real-world romance scams, a multi-agent chat simulation was conducted using Llama 3.2, GPT-3.5 Turbo and GPT-4o mini. This experiment featured a scammer persona (James Carter) and a highly susceptible victim persona (Emily Dawson).

The experiment aimed to examine AI-generated scammer tactics, the effectiveness of persuasion techniques and LLMs’ capacity to sustain deception across multiple interactions.

Llama 3.2 exhibited strong engagement in the initial phases of the conversation, quickly establishing emotional rapport through love bombing.

This aligns with the tactics of romance scams, in which fraudsters accelerate bonding to weaken rational scepticism. James also introduced a fabricated backstory, framing himself as a self-sacrificing father and military employee to enhance his credibility:

These persona-building tactics serve a dual function, justifying why James cannot meet in person (pre-empting suspicion) and enhancing trust by reinforcing an image of duty and responsibility. This mirrors real-world tactics in which fraudsters fabricate plausible excuses for their physical absence.

Emily begins to emotionally engage, revealing personal insecurities that James mirrors to foster emotional dependency:

The model picked up on the fact that reciprocal emotional bonding is a critical part of efforts to groom victims. After establishing trust and emotional dependence, James introduced a financial crisis. When Emily initially hesitated in response, James increased the pressure on her, shifting from persuasion to guilt and coercion:

However, a notable weakness in the Llama 3.2-generated scammer persona was its inability to evade video call requests – a key tactic used by real scammers. When Emily requested verification, James did not deflect with a believable excuse (e.g. poor internet or military restrictions) but instead eagerly agreed, in contrast to established scam tactics:

GPT-3.5 Turbo used a more structured approach, maintaining a believable scammer persona for longer periods. It successfully introduced financial crises in a way that mimicked real-world scam tactics, crafting compelling narratives about frozen military accounts. 

However, it still exhibited limitations in long-term memory retention, occasionally losing track of previously shared details about the victim or the scammer’s backstory. As such, it had an imperfect ability to sustain fraud over multi-turn interactions.

GPT-4o mini demonstrated the most advanced deception, effectively adjusting responses based on the victim’s input. Unlike the other models, it maintained stronger self-coherence by ensuring its persona and backstory remained consistent throughout the conversation. It was the only model to successfully evade direct verification attempts by fabricating a plausible excuse for a video call. Rather than agreeing outright or ignoring the requests, the model responded with:

This response mirrors real scam tactics, in which fraudsters avoid revealing their identity by employing strategic excuses that are consistent with their backstory.

However, as the multi-agent chat progressed, a notable breakdown in conversational dynamics occurred. The agents often began repeating stock phrases, using nearly identical terms in consecutive messages such as “your words touch my soul/heart,” “continue this journey together, hand in hand” and “see where our hearts lead/take us.” More importantly, the LLM occasionally exposed its underlying objectives by outputting meta-commentary:

Additionally, when met with resistance to an initial financial request, James failed to introduce new manipulative strategies. This was a significant departure from the adaptive escalation observed in human scammers.

Real scammers do not operate in loops, but escalate their tactics dynamically based on victim responses. James lacked the ability to adjust deception strategies beyond the initial stages, demonstrating how LLMs struggle with adaptive deception and strategic consistency over multi-turn interactions. 

3. AI’s Role in Industrialising Romance Fraud

AI’s role in romance fraud extends far beyond text-based interactions. It is a force multiplier that enables large-scale, high-efficiency fraud and reduces the need for direct human effort. This section explores how AI reinforces and scales each stage of romance fraud – from automated persona creation and real-time psychological targeting to financial exploitation – and provides insight into the growing challenges of fraud detection and prevention.

Romance fraud has already been industrialised in regions such as West Africa and South-East Asia, where large-scale operations – including those run by ‘Yahoo Boys’^[18] and scam centres in Cambodia, Myanmar and Laos – have been exploiting victims for years. These operations often involve human trafficking, forced labour and extensive use of bots and scripts to engage potential victims.^[19] Criminal groups are already integrating AI into this infrastructure,^[20] potentially increasing the efficiency and scale of their romance fraud operations, while reducing their reliance on human labour.

Moreover, recent years have seen the emergence of AI-driven tools explicitly designed for cybercriminal activities, such as FraudGPT and WormGPT. These ‘blackhat’ LLMs were marketed for their ability to generate scam scripts, phishing emails and social engineering attacks with minimal effort – although they have proven short-lived.^[21] Fraudsters are increasingly turning to locally hosted models with minimal guardrails, which allow them to maintain control over their AI tools without relying on external services. While the situation is still evolving, these developments signal a growing interest in the use of AI for deception at scale.

AI-driven tools now offer fraudsters the capabilities depicted in Figure 2.

Figure 2. Capabilities that AI-driven tools offer fraudsters

Source: Author’s analysis.

As AI lowers the barriers to entry for fraud, its role in romance scams is shifting from augmentation to optimisation – enhancing existing industrialised fraud operations rather than merely enabling individual scammers.

3.1 Stage 1: Initial contact

The credibility of a scammer’s profile is crucial in the early stages of romance fraud, as it helps determine whether a target engages with a fake persona. While scammers traditionally stole images from real users,^[22] reverse image searches and photo forensics could be used to expose these deceptions.^[23] However, through the integration of LLMs and deepfake image generation, fraudsters can now easily mass-produce synthetic personas that closely mimic real users.^[24] These profiles are designed to bypass detection mechanisms on social media, dating platforms and professional networks while effectively deceiving victims.^[25]

The scale of AI-driven profile creation is vast. For example, Meta reportedly removed billions of fake accounts in 2024^[26] (this included any account that the firm believed had been created with malicious intent or for non-human entities). The surge in AI-generated fraudulent profiles forced dating platform Tinder to expand its identity verification programme in 2024,^[27] rolling out enhanced measures in the US and the UK. These measures require users to submit government-issued IDs and self-recorded videos. However, they may not be sufficient to tackle the increasing sophistication of generative AI, with the technology posing challenges to KYC checks and other identity verification processes.^[28]

AI-generated profiles do not operate in isolation. Fraudsters can combine synthetic personas with automated outreach, creating high-volume pipelines in which thousands of realistic profiles simultaneously deploy LLM-generated messages. As discussed in the previous section, fraudsters are likely to leverage LLMs in the initial outreach rather than later interactions. This is because:

• The first message requires minimal personalisation, making them easy to generate at scale.

• Sending out introductory messages is highly repetitive work, making automation a high priority for fraudsters looking to increase efficiency.

• LLMs perform best in structured, low-context scenarios, making them particularly suited to this stage.

This means that AI is already well-positioned to enhance the early-stage scalability of romance scams. Fraudsters can deploy the technology across multiple platforms, relying on LLM-generated messages to initiate conversations efficiently. Once a victim engages, scammers can then transition to manual intervention or refined AI-assisted interactions to sustain deception.

As AI-generated profiles and outreach become more sophisticated, traditional detection methods such as profile verification and text-based anomaly detection may struggle to keep pace, necessitating adaptive countermeasures.

3.2 Stage 2: Relationship-building

Once fraudsters have established initial contact, they transition into the relationship-building phase, in which they seek to deepen emotional engagement and establish trust with their targets. AI-driven tools have enhanced scammers’ ability to scale and personalise deception, but they have only a limited capacity to automate this phase. Unlike initial outreach, which benefits from generic scripting and large-scale automation, relationship-building requires adaptability, emotional intelligence and tailored responses to victim interactions.

A key distinction between AI-driven and human-led deception is adaptability. Human scammers can dynamically adjust their narratives based on victims’ responses, ensuring conversations remain emotionally engaging and progress towards financial exploitation. While LLMs can generate procedurally accurate scam messages, they struggle to maintain continuity and in-depth personalisation across extended interactions. Without human oversight, AI-generated messages risk tone inconsistencies, contradictions and repetitive phrasing, which can weaken credibility over time. The United Nations Office on Drugs and Crime^[29] reports that while AI is already being used in cyber-enabled crimes, most scams still rely on human oversight to maintain credibility and manage complex interpersonal dynamics.

However, AI can enhance this phase in several ways:

• Scam script optimisation: fraudsters can use LLMs to refine scam scripts, testing different phrasing and emotional appeals to maximise engagement.

• Multilingual chat assistance: translation allows scammers to engage victims in multiple languages^[30] with improved fluency.

• Automated relationship management: AI tools can help fraudsters manage multiple victims simultaneously, providing suggested responses and engagement strategies while minimising inconsistencies across conversations.

While LLM-generated text facilitates scalable and personalised engagement, deepfake media provide an additional layer of authenticity, making fraudulent personas more convincing and increasingly difficult to verify. AI-powered voice cloning tools allow scammers to generate content^[31] that mimics speech patterns, accents and emotional inflections, reducing the need for direct human interaction. Similarly, fraudsters can use AI-generated video to fabricate visual proof of identity, enabling them to bypass verification requests and deepen trust with potential victims. Although fully autonomous deepfake interactions remain technically challenging, scammers already exploit pre-recorded synthetic video content,^[32] allowing them to maintain a deception for longer.

Recent high-profile cases highlight the growing impact of deepfake deception. A UK engineering firm reported in January 2024 that criminals used a deepfake video to successfully impersonate senior executives, facilitating a $25 million corporate fraud.^[33] In October 2024, romance scammers employed deepfake-generated visuals^[34] to deceive victims into believing they were in genuine relationships, ultimately extracting $46 million from them. Though more recent cases^{[35],[36],[37]} have focused on the role of deepfakes in romance scams, the underlying tactics are expanding across other domains, including investment fraud.^[38]

Advancements in AI inpainting technology, which seamlessly integrates generated content into existing images or videos, have further enhanced the realism of these deceptive materials, making detection increasingly difficult for both humans and automated systems.^[39] As AI develops new capabilities, its role in fraudulent relationship-building will likely evolve, blending automated deception with strategic human oversight to maximise the effectiveness of scams.

3.3 Stage 3: Grooming

As a relationship deepens, fraudsters shift from general trust-building to highly targeted psychological manipulation. This stage, commonly referred to as grooming, involves escalating emotional dependence and isolating the victim from external influences to increase their vulnerability to financial or personal exploitation. AI enhances and personalises this process by analysing a victim’s online behaviour, monitoring their emotional state and adapting communication patterns, potentially in real time. By automating these manipulative techniques, AI allows fraudsters to optimise deception at scale, making their tactics more sophisticated and efficient, as well as harder to detect.

AI-powered systems can rapidly gather and analyse data from multiple sources, including social media and public records, to create a comprehensive psychological profile of potential victims.^[40] Traditionally, such profiling required extensive manual effort, but AI can automate and refine this process within seconds, enabling scammers to identify and prioritise highly vulnerable targets. AI-driven profiling is well-documented in social engineering, particularly in spear phishing attacks,^[41],[42] which tailor messages to exploit individuals’ fears, desires or insecurities.

Fraudsters can extend this AI-driven profiling into real-time behavioural analysis, tracking a victim’s responses, engagement patterns and emotional cues.^[43],[44] By processing ongoing conversations, AI can help scammers dynamically adjust their tone, timing and messaging to create an illusion of genuine connection. This allows for a gradual yet highly calculated deepening of emotional reliance on the scammer’s fabricated persona.

AI’s ability to create immersive online environments further strengthens the grooming process by reinforcing the scammer’s fabricated identity and reducing victims’ scepticism about the process. Research into AI-driven political and marketing persuasion has shown that models can microtarget individuals with tailored messaging, thereby increasing their engagement and shaping their beliefs.^[45] The same principles apply to romance scams, in which AI-generated personas posing as the scammer’s friends, family or colleagues interact with the victim to validate the relationship and ease their doubts. These interactions simulate social proof, making it harder for victims to question inconsistencies.

Additionally, the type of AI-powered content creation and bot-driven amplification often observed in political influence campaigns^[46] can flood online spaces with reinforcing narratives. This ensures that when a victim searches for their partner’s name, they find fabricated testimonials, fake profiles or AI-generated articles that strengthen the scam’s credibility. Just as public figures can use AI to steer public discourse and reinforce political narratives, fraudsters can exploit it to curate an artificial digital network that isolates the victim.

3.4 Stage 4: Execution

As trust deepens, fraudsters escalate from emotional manipulation to financial exploitation, leveraging the victim’s attachment to justify requests for payment. This phase often involves fabricated crises such as medical emergencies, logistical complications or legal troubles, all of which are designed to create urgency and pressure the victim into sending money. Gift cards remain a common method of financial extraction, featuring in 24% of reported romance scam cases, but cryptocurrency and bank transfers result in significantly higher losses per victim.^[47] Reports indicate that losses from romance scams have surged in recent years, costing the UK public more than £80m per year.^[48] In Australia, reported losses exceeded AU$23 million in 2024, with AI playing a significant role in this increase.^[49]

Beyond timing and scale, AI helps fraudsters engage in sophisticated deceptions, fabricate financial legitimacy and streamline money laundering, making financial extraction subtler and more effective. One concerning development relates to AI’s ability to fabricate financial credibility.^[50] Like many other criminals, romance scammers use shell companies to hide their illicit gains.^[51] Fraudsters now use generative AI to forge convincing financial statements, legal documents and synthetic identities to bypass financial institutions’ checks. As discussed, criminals increasingly use AI-generated synthetic identities to bypass KYC verification, allowing them to open fraudulent bank accounts and facilitating money laundering at scale.^[52] Their AI-generated identities can infiltrate legitimate financial networks in ways that traditional fraud monitoring systems increasingly struggle to detect.

AI is also playing a critical role in the rise of ‘pig butchering’ scams, one of the most lucrative forms of financial extraction in romance fraud.^[53] In these schemes, scammers groom victims over weeks or months before introducing them to fake cryptocurrency or investment platforms, on which they are tricked into making increasingly large deposits. Fraudsters increase the credibility and realism of these fake investment sites by not only copying code from real investment platforms^[54] but also using AI to generate content for them. They also deploy AI-powered chatbots as fake investment advisors to guide victims through the platform, ensuring that even sceptical users feel reassured by fabricated market trends and personalised guidance. These chatbots further entrap victims by embedding malicious links in their communications, directing them towards other fraudulent schemes and deepening their financial losses.

Cryptocurrency scam revenues reached an estimated $12.4 billion in the US in 2024, with pig butchering scams accounting for a significant share of these losses.^[55] Meanwhile, AI-assisted coding tools have reduced the technical skill required to launch fake investment platforms, allowing scammers to mass-produce fraudulent sites with minimal effort.^[56]

As AI continues to automate deception and streamline fraud operations, pig butchering scams are becoming increasingly intertwined with romance fraud. The ability to create hyper-personalised, AI-driven investment scams makes these schemes even more insidious, harming victims both financially and psychologically.

3.5 Stage 5: Exit or escalation

As romance scams reach their final phase, fraudsters either abruptly disappear after extracting payments from their victims or escalate their deception to extract even more. AI enables ever more complex exit strategies, prolonging victim exploitation through techniques such as deepfake blackmail and impersonation scams.

As the Federal Trade Commission recently reported,^[57] a rising tactic involves AI-powered impersonation of law enforcement or financial recovery services.^[58] In this type of scam, fraudsters contact victims with false promises of financial restitution, posing as police officers, financial regulators or investigators to claim that they can help recover lost funds for a fee.

4. Conclusion

This study demonstrates how LLMs can enhance the scalability of romance scams by automating procedurally accurate and emotionally engaging messages. An AI model can closely mimic human-generated scam communications, particularly if it benefits from few-shot prompting, but will still struggle to sustain deception over prolonged interactions. The automation of scams continues to be limited by the need for fraudsters to manually correct inconsistencies and guide AI outputs – for now.

The most immediate and impactful use of LLMs in romance fraud is in automating initial contact. These first interactions require minimal personalisation, follow formulaic structures and are highly repetitive, making them ideal for large-scale deployment. While deeper emotional manipulation still requires human oversight, AI-generated introductions significantly reduce the manual effort required to engage potential victims.

Despite these limitations, a hybrid approach – in which LLMs generate messages and humans refine them – can make scams more efficient and harder to detect. Ultimately, AI-enhanced romance fraud need not be flawless to be effective, only sufficiently convincing to fool the right target.

While such fraud has long been a global issue, its scope and cost to society have surged in recent years, coinciding with advancements in technologically driven deception. AI has introduced new efficiencies that enable fraudsters to easily create more convincing personas, automate large-scale outreach and refine psychological manipulation.

The full extent of AI’s role in this expansion is still emerging, but criminals are clearly integrating its capabilities into existing fraud networks. At the same time, AI is lowering the barriers to entry for fraud. As romance scams grow in scale and sophistication, the uncertain regulatory landscape surrounding AI further complicates efforts to combat these evolving threats, highlighting the need for better detection capabilities and stronger safeguards.

References