Shortcuts
General AI
AI Safety and Transparent and Explainable AI (T/XAI):
Foundation Model Transparency Index (FMTI)
Fair-LLM-Benchmark ("Bias and Fairness in Large Language Models: A Survey" 2024)
Aequitas: Bias Auditing & "Correction" Toolkit
(KDD 2023 Tutorial)
Data and code of the paper "Dissecting racial bias in an algorithm used to manage the health of populations" (Science Magazine 2019)
(XAITK-Saliency)
XNLP: XAI for Natural Language Processing
AI Art Generators: AIGCBench Ziqi Huang's Awesome Evaluation of Visual Generation
Open AI's DALL·E 2
Bing Image Creator
Stability AI
(Stable Diffusion,
Visual ChatGPT)
Midjourney
Dream by WOMBO
Natural Language Processing and Computational Linguistics
LLMs: 🤗 Open LLM Leaderboard LLM-Leaderboard (community-based) Awesome Machine Generated Text A Survey on Language, Multimodal, and Scientific GPT Models: Examing User-Friendly and Open-Sourced Large GPT Models Prompt Engineering Guide
Min Woo (Daniel) Park's Open-LLM-datasets
PleIAs's Common Corpus
PleIAs's OpenCulture
PleIAs's Toxic Commons
PleIAs's Finance Commons
Awesome-Medical-Healthcare-Dataset-For-LLM
Awesome papers on LLMs detection
A Survey of Attributions for Large Language Models (2023)
Factcheck-GPT (2023)
DetectGPT (2022)
General Tools:
NLTK (Natural Language Toolkit)
spaCy
(
)
(GitHub)
Natural
CogCompNLP
Hugging Face
(datasets;
Write With Transformer)
Talk to Transformer (InferKit online demo)
quanteda: Quantitative Analysis of Textual Data in R
(GitHub)
gensim – Topic Modelling in Python
RNNTagger
TreeTagger
Python Word Segmentation
Word Ninja
SymSpell
(Python port: symspellpy)
Language Style Transfer (NIPS 2017)
GeoTxt (Transactions in GIS 2019)
Edinburgh Geoparser
GeoPy
XAI for Natural Language Processing (AACL-IJCNLP 2020)
Google's BERT
(GitHub)
(WuDaoCorpora;
GitHub,
GLM,
CLM;
BMInf)
BERTective (EACL 2021)
mauve-experiments (NeurIPS 2021)
Chinese NLP Resources: 预训练模型仓库 百度ERNIE Pretrained language model and its related optimization techniques developed by Huawei Noah's Ark Lab (鹏程.盘古α / PanGu-α)
awesome-chinese-nlp (Guan Wang)
“结巴”中文分词
THUAIPoet (九歌) research group
(九歌V2.0;
BERT-CCPoem,
MixPoet @ AAAI 2020,
Stylistic Poetry @ EMNLP 2018,
WMPoetry @ IJCAI 2018;
中国古典诗歌匹配数据集 / CCPM = Chinese Classical Poetry Matching Dataset,
Other datasets)
少女诗人小冰
tensorflow_poems / LiBai AI Composer / 中文古诗自动作诗机器人
中文语料小数据
Datasets: Nicolas Iderhoff's nlp-datasets WordNet Wikimedia Downloads
(Frequency lists)
WordNet
Amazon MASSIVE dataset
WebNLG Challenge
Wiktextract
(data @ kaikki.org)
Use of corpora in translation studies @ Centre for Translation Studies, University of Leeds
OpenLexicon
Lexique
(WorldLex: Blog, Twitter and Newspapers Word Frequencies for 66 languages)
Datasets of Automatic Keyphrase Extraction @ LIAAD, INESCTEC
KPTimes Corpus @ INLG 2019
dewiki-wordrank
OAGSX Title Generation Dataset
OAGKX Keyword Generation Dataset
GeoNames
Awesome LLM-generated Text Detection (2023)
Awesome papers on LLMs detection (2023)
M4: Multi-generator, Multi-domain, and Multi-lingual Black-Box Machine-Generated Text Detection (2023)
Privacy-related resources:
(PrivaSeer Corpus @ ACL 2021,
PrivBERT @ ACL 2021)
Federated Learning
General Resources:
Awesome-Federated-Learning
The Federated Learning Portal
Open-source Tools: TensorFlow Federated (TFF) (GitHub) NVIDIA Clara FedML: A Research Library and Benchmark for Federated Machine Learning
(GitHub)
(Federated Learning Research at Webank AI)Commercial Solutions:
