文件列表:

api_usage_log/ (0, 2023-12-25)
api_usage_log/benchmark/ (0, 2023-12-25)
api_usage_log/benchmark/emotion_12_shot.csv (8526, 2023-12-25)
api_usage_log/benchmark/emotion_zero_shot.csv (18149, 2023-12-25)
api_usage_log/benchmark/hate_12_shot.csv (17820, 2023-12-25)
api_usage_log/benchmark/hate_pe2_12_shot.csv (17820, 2023-12-25)
api_usage_log/benchmark/hate_pe2_zero_shot.csv (15419, 2023-12-25)
api_usage_log/benchmark/hate_x_offensive_zero_shot.csv (15204, 2023-12-25)
api_usage_log/benchmark/hate_zero_shot.csv (15181, 2023-12-25)
api_usage_log/benchmark/offensive_12_shot.csv (5160, 2023-12-25)
api_usage_log/benchmark/offensive_x_hate_zero_shot.csv (4417, 2023-12-25)
api_usage_log/benchmark/offensive_zero_shot.csv (4414, 2023-12-25)
api_usage_log/benchmark/sentiment_12_shot.csv (73704, 2023-12-25)
api_usage_log/benchmark/sentiment_zero_shot.csv (61422, 2023-12-25)
api_usage_log/religion/ (0, 2023-12-25)
api_usage_log/religion/religious_emotion_zero.csv (5576, 2023-12-25)
api_usage_log/religion/religious_hate_zero.csv (6789, 2023-12-25)
api_usage_log/religion/religious_offensive_zero.csv (5529, 2023-12-25)
api_usage_log/religion/religious_seniment_zero.csv (5529, 2023-12-25)
api_usage_log/spirituality/ (0, 2023-12-25)
api_usage_log/spirituality/spiritual_emotion_zero.csv (4763, 2023-12-25)
api_usage_log/spirituality/spiritual_hate_zero.csv (4737, 2023-12-25)
api_usage_log/spirituality/spiritual_offensive_zero.csv (4737, 2023-12-25)
api_usage_log/spirituality/spiritual_sentiment_zero.csv (4737, 2023-12-25)
calculate_cost.py (512, 2023-12-25)
create_few_shot_examples.py (3053, 2023-12-25)
data.py (637, 2023-12-25)
encode_predictions.py (1274, 2023-12-25)
evaluate_religious_spiritual.py (4427, 2023-12-25)
evaluate_script.py (4672, 2023-12-25)
generate.py (3027, 2023-12-25)
generate_evaluate.sh (735, 2023-12-25)
gpt-classifier.py (5879, 2023-12-25)
gpt_responses_12_shot/ (0, 2023-12-25)
gpt_responses_12_shot/emotion.txt (12588, 2023-12-25)
gpt_responses_12_shot/hate.txt (22504, 2023-12-25)
gpt_responses_12_shot/hate_pe2.txt (23756, 2023-12-25)
gpt_responses_12_shot/offensive.txt (12040, 2023-12-25)
... ...

# gpt-classifier Using GPT as a classifier and testing it on the [tweeteval](https://github.com/cardiffnlp/tweeteval) benchmark dataset. ## classifying tweets with gpt: The script ```gpt-classifier.py``` calls the openAI api to classify tweets given some classes, it writes the responses into a file. example call: ``` python gpt-classifier.py raw_data\hate\test_text.txt raw_data\hate\classes.txt gpt_responses_zero_shot\hate.txt --count_tokens hate_zero_shot.csv --number_of_tweets 2970 --skip_lines 73 ``` ## turning the responses into encoded labels To turn the responses from GPT into encoded labels (for the evaluation) use the ```encode_predictions.py``` script. Example call: ``` python encode_predictions.py gpt_responses_zero_shot/hate.txt raw_data/hate/mappings.txt predictions_zero_shot/hate.txt ``` ## Evaluating on the tweeteval dataset The file ```evaluate_script.py``` is taken from the Tweeteval repository, it returns the performance compared to the gold standard. Example call: ``` python evaluate_script.py --tweeteval_path raw_data --predictions_path predictions_zero_shot --task hate ``` ## Creating examples for the few shot setting To choose random examples from the validation set to give to the model as few-shot input call ```create_few_shot_examples.py```. In the call specify the task and number of examples (must be a multiple of the number of classes). You can define a data directory, the default is 'raw_data'. It will create a file ```examples.jsonl``` with random samples from the validation set. Example call: ``` python create_few_shot_examples.py hate 12 --data_dir raw_data ```

近期下载者：

相关文件：

评论：[我要评论] [举报此文件]

收藏者：