How do anti-phishing systems survive sneaky URL attacks?

URL phishing detectors protect users daily, but how robust are they when attackers rewrite addresses? This study creates a comprehensive test to measure anti-phishing resistance. Four models—logistic regression, XGBoost, CharCNN, and BERT-base—are evaluated on over 235,000 real URLs, including unseen attack families. Models score excellently on clean data, but many collapse under realistic stress. Subdomain injection drops performance to around 0.43 for some models, while BERT is highly vulnerable to homoglyph, padding, and path-based tricks. Transfer tests show poor generalisation to new attack families, and explanations (e.g., SHAP) become unstable. These findings provide a solid benchmark for building more reliable anti-phishing tools that can face evolving threats.

The protocol integrates five critical dimensions: structured adversarial perturbations, unseen attack‑family generalisation, compositional attack effects, explanation stability, and external vulnerability transfer. For adversarial perturbations, subdomain injection, homoglyph, padding, and path‑based attacks are applied and measured against baseline accuracy. Unseen‑family generalisation uses a leave‑one‑family‑out scheme, computing the Robustness Degradation Index (RDI). Compositional effects test multiple perturbations simultaneously. Explanation stability employs top‑K SHAP Jaccard similarity between clean and perturbed inputs. External transfer validates models on an independent PhishTank phishing set. Results show traditional models (LR, XGBoost) collapse under subdomain injection, while transformer‑based BERT is highly sensitive to character‑ and path‑based tricks. RDI scores reveal poor generalisation to new families, and Jaccard drops indicate unstable explanations. This integrated framework offers a realistic, comprehensive benchmark for robust phishing‑URL detection.