Open Source Software Engineer

Job not on LinkedIn

🔥 0 minutes ago

🗽 New York – Remote

🗽 New York – Remote

💵 $60 - $80 / hour

⏱ Part Time

🟡 Mid-level

🟠 Senior

🧑‍💻 Full-stack Engineer

👻 Ghost score 0%

What it means: how likely this posting is sitting open without active hiring.How to read it: 60% or higher means we think this is a ghost. Lower scores are less of a worry.How we score it: age, reposts, open roles vs team size, how fast jobs close, and whether pay is listed.Fine print: an estimate from public data, not a judgment of the company. Real jobs can score high, so treat it as a hint, not a fact.

Apply Now

Find Similar Remote Jobs

Set Up Email Alert For New Jobs

📊 Check your resume score for this job

Improve your chances of getting an interview by checking your resume score before you apply.

Upload Resume

24-MAG

WebsiteLinkedInAll Job Openings

2 - 10 employees

🤝 B2B

💼 Consulting

B2B • Consulting

24-MAG is a commercial strategy and execution firm that helps B2B organizations design and implement systems, workflows, and operating rhythms for sales, client management, and cross-functional projects. They focus on transforming scattered processes into aligned, measurable, and scalable commercial functions—covering pipeline structure, account management frameworks, and operational discipline for teams seeking efficient, intentional growth.

📋 Description

• Review software-engineering benchmark tasks for correctness, reproducibility, and grading integrity • Audit repository-level assignments, reference patches, test harnesses, containerised environments, and evaluation logic • Assess task requirements, repository state, dependencies, configuration, expected behaviour, and reproducibility • Review reference patches for correctness and completeness and identify unintended behavioural changes or incomplete fixes • Audit test runners and automated grading logic for coverage, brittle assertions, and scoring consistency • Validate dependency installation, builds, configuration, runtime requirements, Docker isolation, and containerised execution • Identify answer leakage, unintended shortcuts, reward-hacking opportunities, loopholes, and exploitable behaviours • Investigate failing or inconsistent benchmark tasks using stack traces, logs, test failures, and repository behaviour • Apply open-source contribution and maintainer experience to evaluate repository conventions, patches, and workflows • Evaluate Python software and tasks involving Java, Go, TypeScript, or C++ • Assess benchmark tasks against structured technical criteria and provide precise written explanations with code, test, patch, or execution references • Distinguish substantive benchmark defects from minor implementation differences and recommend improvements

🎯 Requirements

• 3+ years of professional software engineering experience • Demonstrated open-source contribution or maintainer experience, such as merged pull requests, committer responsibilities, or maintainer roles • Strong ability to review repository-level software changes • Experience auditing reference patches, test runners, and automated test suites • Comfortable evaluating Docker-based isolation and reproducible development environments • Strong understanding of software testing, debugging, and code-review practices • Ability to identify answer leakage, reward hacking, or other benchmark-integrity issues • Strong proficiency in Python • Professional fluency in at least one additional language such as Java, Go, TypeScript, or C++ • Familiarity with SWE-Bench Verified or similar repository-level software engineering benchmarks is preferred • Maintainer or contributor history with established Python open-source projects is highly valued • Previous code-review, software evaluation, or task-grading experience is advantageous • Strong written communication and ability to provide precise technical feedback • Work must be completed without using confidential or proprietary information belonging to any employer, client, institution, or other third party • H1-B and STEM OPT support is unavailable for this engagement

🏖️ Benefits

• Part-time independent contractor engagement • Fully remote within the United States • Flexible scheduling based on project requirements • Projects may be extended, shortened, or concluded based on project needs and performance