Deteksi Ucapan yang Dihasilkan oleh Artificial Intelligence Menggunakan Metode Convolutional Neural Network
DOI:
https://doi.org/10.53513/jursi.v5i3.10434Keywords:
Audio Deepfake Detection, Partial Fake, Mel-Frequency Cepstral Coefficients, Convolutional Neural NetworksAbstract
Seiring berkembangnya teknologi Artificial Intelligence (AI), khususnya teknologi sintesis suara seperti Text to Speech dan Voice Conversion, media yang dihasilkan oleh teknologi ini semakin realistis, sehingga sulit dibedakan antara suara manusia dan suara yang dihasilkan oleh AI. Fenomena ini mengakibatkan peningkatan signifikan dalam kasus penyalahgunaan, sehingga menimbulkan tantangan serius di berbagai sektor, baik dalam ranah financial technology (fintech) maupun non-fintech. Pada penelitian ini, dalam upaya mendeteksi ucapan hasil manipulasi AI, atau deepfake, digunakan dataset Real, Fake, and Partially fake. Seluruh data pada dataset yang mengandung suara AI digabungkan dan diberi label 'Fake', sedangkan suara manusia diberi label 'Real'. Pendeteksian ini menggunakan pendekatan ekstraksi fitur audio Mel-Frequency Cepstral Coefficients (MFCC), diikuti dengan pemrosesan seperti normalisasi fitur, padding, labelling, dan reshape input. Selanjutnya, data yang telah diproses akan dianalisis menggunakan arsitektur Convolutional Neural Networks (CNN) yang diusulkan. Hasil penelitian ini menunjukkan bahwa model arsitektur CNN yang diusulkan dapat mencapai akurasi pelatihan sebesar 99% pada data training dan validation, serta akurasi pengujian pada data testing sebesar 92% serta untuk hasil evaluasi metrik (precision, recall, F1-score, macro avg, dan weighted avg) model pada data testing cukup memuaskan, yang mana untuk setiap metrik menghasilkan lebih dari 85%, Equal Error Rate (EER) bernilai 8% serta Area Under Precision-Recall Curve (AUPRC) pada kelas ‘Real’ bernilai 89% dan ‘Fake’ bernilai 99% dan hasil sistem yang dikembangkan dapat membedakan hasil suara AI (Fake) dengan suara manusia (Real).
References
I. J. Goodfellow dkk., “Generative Adversarial Networks,” 10 Juni 2014, arXiv: arXiv:1406.2661. Diakses: 4 Juni 2024. [Daring]. Tersedia pada: http://arxiv.org/abs/1406.2661
D. P. Kingma dan M. Welling, “Auto-Encoding Variational Bayes,” 10 Desember 2022, arXiv: arXiv:1312.6114. doi: 10.48550/arXiv.1312.6114.
D. Dagar dan D. K. Vishwakarma, “A literature review and perspectives in deepfakes: generation, detection, and applications,” Int. J. Multimed. Inf. Retr., vol. 11, no. 3, hlm. 219–289, Sep 2022, doi: 10.1007/s13735-022-00241-w.
C. Herke, “Deepfake,” Futuro, 2023, [Daring]. Tersedia pada: https://api.semanticscholar.org/CorpusID:264316779
A. Łańcucki, “FastPitch: Parallel Text-to-speech with Pitch Prediction,” 16 Februari 2021, arXiv: arXiv:2006.06873. doi: 10.48550/arXiv.2006.06873.
J. Kim, S. Kim, J. Kong, dan S. Yoon, “Glow-TTS: A Generative Flow for Text-to-Speech via Monotonic Alignment Search,” 22 Oktober 2020, arXiv: arXiv:2005.11129. doi: 10.48550/arXiv.2005.11129.
J. Shen dkk., “Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictions,” 15 Februari 2018, arXiv: arXiv:1712.05884. doi: 10.48550/arXiv.1712.05884.
B. van Niekerk, M.-A. Carbonneau, J. Zaïdi, M. Baas, H. Seuté, dan H. Kamper, “A Comparison of Discrete and Soft Speech Units for Improved Voice Conversion,” 8 Juni 2022. doi: 10.1109/ICASSP43922.2022.9746484.
LiuSongxiang, CaoYuewen, WangDisong, WuXixin, LiuXunying, dan MengHelen, “Any-to-Many Voice Conversion With Location-Relative Sequence-to-Sequence Modeling,” IEEEACM Trans. Audio Speech Lang. Process., 2021, doi: 10.1109/TASLP.2021.3076867.
V. Popov, I. Vovk, V. Gogoryan, T. Sadekova, M. Kudinov, dan J. Wei, “Diffusion-Based Voice Conversion with Fast Maximum Likelihood Sampling Scheme,” 4 Agustus 2022, arXiv: arXiv:2109.13821. doi: 10.48550/arXiv.2109.13821.
J. Li, W. Tu, dan L. Xiao, “Freevc: Towards High-Quality Text-Free One-Shot Voice Conversion,” dalam ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Rhodes Island, Greece: IEEE, Jun 2023, hlm. 1–5. doi: 10.1109/ICASSP49357.2023.10095191.
D. Katanich, “It’s a scam! How deepfakes and voice cloning taps into your cash,” euronews. Diakses: 4 Juni 2024. [Daring]. Tersedia pada: https://www.euronews.com/business/2024/04/10/its-a-scam-how-deepfakes-and-voice-cloning-taps-into-your-cash
D. Milmo, “Company worker in Hong Kong pays out £20m in deepfake video call scam,” The Guardian, 5 Februari 2024. Diakses: 5 Juni 2024. [Daring]. Tersedia pada: https://www.theguardian.com/world/2024/feb/05/hong-kong-company-deepfake-video-conference-call-scam
“CEO of world’s biggest ad firm targeted by deepfake scam | Technology | The Guardian.” Diakses: 5 Juni 2024. [Daring]. Tersedia pada: https://www.theguardian.com/technology/article/2024/may/10/ceo-wpp-deepfake-scam
M. Cerullo, “AI scams mimicking voices are on the rise,” CBS News. Diakses: 4 Juni 2024. [Daring]. Tersedia pada: https://www.cbsnews.com/news/ai-scam-voice-cloning-rising/
A. Bunn, “Artificial Imposters—Cybercriminals Turn to AI Voice Cloning for a New Breed of Scam,” McAfee Blog. Diakses: 4 Juni 2024. [Daring]. Tersedia pada: https://www.mcafee.com/blogs/privacy-identity-protection/artificial-imposters-cybercriminals-turn-to-ai-voice-cloning-for-a-new-breed-of-scam/
Z. Wu dkk., “ASVspoof 2015: the first automatic speaker verification spoofing and countermeasures challenge,” dalam Interspeech 2015, ISCA, Sep 2015, hlm. 2037–2041. doi: 10.21437/Interspeech.2015-462.
X. Wang dkk., “ASVspoof 2019: A large-scale public database of synthesized, converted and replayed speech,” 14 Juli 2020, arXiv: arXiv:1911.01601. doi: 10.48550/arXiv.1911.01601.
J. Yamagishi dkk., “ASVspoof 2021: accelerating progress in spoofed and deepfake speech detection,” 1 September 2021, arXiv: arXiv:2109.00537. doi: 10.48550/arXiv.2109.00537.
R. Reimao dan V. Tzerpos, “FoR: A Dataset for Synthetic Speech Detection,” dalam 2019 International Conference on Speech Technology and Human-Computer Dialogue (SpeD), Timisoara, Romania: IEEE, Okt 2019, hlm. 1–10. doi: 10.1109/SPED.2019.8906599.
H. Khalid, S. Tariq, M. Kim, dan S. S. Woo, “FakeAVCeleb: A Novel Audio-Video Multimodal Deepfake Dataset,” 1 Maret 2022, arXiv: arXiv:2108.05080. doi: 10.48550/arXiv.2108.05080.
“A dataset of histograms of original and fake voice recordings (H-Voice) - ScienceDirect.” Diakses: 7 Juni 2024. [Daring]. Tersedia pada: https://www.sciencedirect.com/science/article/pii/S2352340920302250?via%3Dihub
Z. Almutairi dan H. Elgibreen, “A Review of Modern Audio Deepfake Detection Methods: Challenges and Future Directions,” Algorithms, vol. 15, no. 5, hlm. 155, Mei 2022, doi: 10.3390/a15050155.
A. AlAli dan G. Theodorakopoulos, “An RFP dataset for Real, Fake, and Partially fake audio detection,” 26 April 2024, arXiv: arXiv:2404.17721. doi: 10.48550/arXiv.2404.17721.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Jurnal Sistem Informasi Triguna Dharma (JURSI TGD)
This work is licensed under a Creative Commons Attribution 4.0 International License.
