Assignment 4: Prompt Engineering Experiments

CSC 375/575 - Generative AI | Fall 2025

Prof. Rongyu Lin, Quinnipiac University

Assignment Overview

Goal: Apply and compare multiple prompt engineering techniques from Lectures 10-11 to systematically improve your Final Project's LLM performance through hands-on experimentation.

Important: This assignment is submitted WITH your Final Project

Assignment 4 deliverables will be included in your Final Project submission on December 12. You do not need to submit this assignment separately. This assignment helps you apply advanced prompting techniques to enhance your Final Project.

What You'll Do:

Select and implement 3-4 prompt engineering techniques from Lectures 10-11
Design specific prompt templates for each technique tailored to your Final Project
Run systematic experiments on 5-10 representative test cases
Compare results and analyze which techniques work best for your use case
Write a comprehensive report documenting your entire experiment

Format: Individual or team (same as your Final Project)

Estimated Time: 8-10 hours

Submission: December 12, 2025 (with Final Project)

Language Models You Can Use

You may use any of the following LLMs for your experiments:

GitHub Copilot: Available through your student account
Ollama: Open-source models running locally (e.g., Llama 3, Mistral, CodeLlama)
Other open-source models: Any publicly available LLM you can access
Commercial APIs: ChatGPT, Claude, or other services (if you have access)

Note: Document which model(s) you used in your report.

Prompt Engineering Techniques to Choose From

Select 3-4 techniques from Lectures 10-11. You may choose from the list below or use other techniques covered in class:

Available Techniques (Including but not limited to)

Chain-of-Thought (CoT): Step-by-step reasoning prompts
Few-Shot Learning: Providing 2-5 example input-output pairs
Zero-Shot CoT: "Let's think step by step" prompting
Self-Consistency: Sampling multiple reasoning paths and voting
Problem Decomposition: Breaking complex tasks into subtasks
Role Prompting: Assigning specific roles or personas to the model
Instruction Refinement: Iteratively improving prompt clarity and specificity
Output Format Control: Specifying structured output formats (JSON, tables, etc.)
Other techniques from Lectures 10-11

Selection Criteria:

Choose techniques that are relevant to your Final Project's task
Consider diversity in approach (e.g., one reasoning-based, one example-based)
Ensure you can implement and test each technique meaningfully

Deliverable: Experiment Report (100 points)

Submit one comprehensive PDF report documenting your complete prompt engineering experiment.

Report: prompt_engineering_report.pdf

Length: No page limit - write as much as needed to fully document your work

Focus on completeness and clarity, not page count.

Your report should be comprehensive enough that someone else could reproduce your experiments. Include all necessary details about techniques, prompts, test cases, and results.

Report Structure (3 Parts)

Part 1: Technique Selection & Prompt Design (30 points)

Which 3-4 techniques you selected and why
Complete prompt templates for each technique
Design rationale explaining your choices

Part 2: Experimental Implementation & Results (40 points)

Your 5-10 test cases and what they evaluate
Experimental results with comparison tables
Example outputs from each technique

Part 3: Analysis & Reflection (30 points)

Which techniques performed best and why
What you learned about prompt engineering
How you will apply these findings to your Final Project

Deliverable

prompt_engineering_report.pdf

Submission

Submit with Final Project on December 12, 2025

Include prompt_engineering_report.pdf in your Final Project submission.

Grading Rubric (100 points)

Component	Points	Criteria
Technique Selection & Prompt Design	30	Appropriate selection of 3-4 techniques (8 pts) Clear and complete prompt templates (12 pts) Strong design rationale (10 pts)
Experimental Implementation & Results	40	Quality and diversity of test cases (10 pts) Completeness of experiments (15 pts) Clear results presentation with tables and examples (15 pts)
Analysis & Reflection	30	Depth of comparative analysis (12 pts) Quality of insights and reflection (10 pts) Practical application to Final Project (8 pts)

Back to Course Page