Skip to content

remove-lm-mask #175

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
david-thrower opened this issue Apr 12, 2025 · 1 comment
Open

remove-lm-mask #175

david-thrower opened this issue Apr 12, 2025 · 1 comment
Assignees
@david-thrower
Labels
close/do-not-merge Failed attempts or other dead ends hold/more-ml-validation-testing-required More scientific validation require, usually applied due to ambiguous results in small sample tests.

Comments

@david-thrower
Copy link
Owner

It appears a phishing_email_detection_gpt2.py line 407 needs the zero mask set to False. This is an AI - introduced error.

embedded = tf.keras.layers.Embedding(
    input_dim=VOCABULARY_SIZE,
    output_dim=EMBEDDING_DIM,
    input_length=max_seq_length,
    mask_zero=True)(tokens)
@david-thrower david-thrower added triage/high-priority audience/technical Issue primarily for technical review and service. kind/performance labels Apr 12, 2025
@david-thrower david-thrower self-assigned this Apr 12, 2025
@david-thrower
Copy link
Owner Author

Do not merge

Trial #     val_binary_accuracy
Trial 1:    0.9499 
Trial 2:    0.9387

Where at least 0.952 was expected, See https://github.com/david-thrower/cerebros-core-algorithm-alpha/actions/runs/14415528912/job/40431381425 (0.97 ) and https://github.com/david-thrower/cerebros-core-algorithm-alpha/releases/tag/v0.10.0-alpha. This change should be tested further, just to see if this face value appearance of performance degradation was spurious, which it may be with a sample size of 2. For now, it appears to be unwise to merged this until there is evidence to discredit this possibility.

@david-thrower david-thrower added close/do-not-merge Failed attempts or other dead ends hold/more-ml-validation-testing-required More scientific validation require, usually applied due to ambiguous results in small sample tests. and removed triage/high-priority audience/technical Issue primarily for technical review and service. kind/performance labels Apr 13, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@david-thrower david-thrower

Labels
close/do-not-merge Failed attempts or other dead ends hold/more-ml-validation-testing-required More scientific validation require, usually applied due to ambiguous results in small sample tests.
Projects
None yet
Milestone
No milestone
Development

When branches are created from issues, their pull requests are automatically linked.

refs/heads/175-remove-lm-mask david-thrower/cerebros-core-algorithm-alpha
1 participant
@david-thrower
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy