Classroom-Scale Bayesian Knowledge Tracing under AI-Assisted Assessment

By Abhinash Jena on August 27, 2026

This article is an empirical classroom implementation of an adjusted Bayesian Knowledge Tracing (BKT) method in a 120-hour Business Analytics training course delivered at GIET, Sonipat. The course used continuous quiz-based assessment alongside attendance, reporting, feasibility analysis, VIVA, and capstone evidence. Because students were permitted to use AI while taking quizzes, correctness could not be interpreted as a pure indicator of independent mastery. The study therefore operationalised BKT as an interpretable classroom-scale model rather than as a fully fitted platform-scale learner model. Item accuracy was smoothed using a pseudo-count correction, the guess parameter was adjusted using item answerability, and slip and learning probabilities were adjusted using calibrated item difficulty. Final mastery was classified into four pedagogical bands.

Pedagogical Classification Bands
Pedagogical Classification Bands

The empirical dataset consisted of a response matrix for 39 students across 407 administered quiz items, producing 8,209 attempted responses, and a labelled question bank of 586 items tagged by component, module, skill, and Bloom level. The goal was to evaluate whether the implemented model behaved pedagogically by examining item discrimination, warning-pattern items, Bloom-level calibration, and correlations between quiz accuracy, engagement volume, model-based ability, and the broader achievement composite.

Of the 366 items with computable discrimination, 46.2% showed good discrimination, 23.5% marginal discrimination, and 30.3% poor discrimination. A warning pattern – high pass rate combined with low discrimination – appeared in 15.3% of computable items and clustered more strongly in mechanical Excel and procedural skills than in judgement-oriented business requirement items. Bloom-level labels did not show a significant monotonic relationship with observed pass rate or discrimination. Ability estimates correlated more strongly with quiz engagement volume than with quiz accuracy, while raw quiz accuracy was a weak and non-significant predictor of the multi-source achievement composite. These findings support the use of adjusted BKT as a formative, pedagogical diagnostic under AI-assisted assessment, while also showing that quiz correctness must be interpreted cautiously and validated against broader evidence of learning.

Re-interpretation of Quiz Correctness

Knowledge tracing models a learner’s evolving mastery of specific skills from sequences of observed responses. Bayesian Knowledge Tracing (BKT) remains attractive for classroom settings because its logic is transparent: each skill is represented through initial mastery P(L0), learning or transition P(T), slip P(S), and guess P(G), and the learner’s mastery estimate is updated after each attempt. This interpretability makes BKT useful for teachers who need actionable diagnostic information rather than only predictive accuracy.

However, contemporary classroom assessment increasingly takes place under open-book or AI-assisted conditions. In such settings, a correct answer may reflect independent mastery, but it may also reflect search, prompt use, recognition, peer help, or successful verification through a generative AI tool. This does not make quiz data unusable, but it changes the meaning of correctness. The central measurement problem is no longer only whether a student answered correctly; it is whether the observed response is strong evidence of independent skill mastery.

The Business Analytics course was deliberately designed to address this measurement challenge. Students were permitted to use AI during quizzes, while their learning was assessed through multiple complementary components, including reporting quality, feasibility analysis, VIVA, and capstone performance.

Study design
Study design

By combining AI-assisted quiz activity with applied and performance-based evidence, the course design provided an appropriate setting for examining whether a classroom-scale BKT adjustment could yield useful pedagogical diagnostics without equating quiz correctness with independent mastery.

Furthermore, this article reframes BKT as a defensible classroom diagnostic rather than as a fully fitted intelligent tutoring model. It asks whether BKT parameters can be adjusted using transparent educational assumptions about answerability, difficulty, Bloom-level, and response evidence. Also, whether the resulting mastery estimates can support session planning, remediation, consolidation, and advancement decisions.

The study makes three contributions.

  1. It demonstrates how BKT can be operationalised at classroom scale using instructor-controlled parameter adjustments rather than full expectation-maximisation estimation, a standard method for estimating model parameters when the true mastery state is hidden.
  2. It shows how item discrimination, Bloom-level calibration, and correlation with broader course outcomes can be used to evaluate whether quiz correctness is diagnostically meaningful under AI-assisted conditions.
  3. It converts probabilistic mastery estimates into pedagogical categories – Relearn, Practice, Consolidate, and Advance – that can be used for session planning.

Business Analytics Course Context and Data Source

The dataset comes from a Business Analytics Training course delivered at GIET, Sonipat, Haryana, in July 2026 by Project Guru in association with Symbiosis. The course introduced spreadsheet-based analytics, data structuring, dashboard interpretation, and business problem framing across approximately 120 contact hours. Its instructional objective was to move students from surface-level understanding of business problems toward structured analytical thinking:

  1. identifying stakeholders,
  2. framing requirements,
  3. designing data structures,
  4. interpreting dashboards, and
  5. supporting decisions using evidence.

The assessment was continuous rather than terminal. Furthermore, students were evaluated on a 40-point scale using quiz performance, attendance, reporting, feasibility work, capstone outputs, technical practice activities, and VIVA. The empirical unit of analysis in this article is the student-question quiz response. Three data sources were used:

  • Response matrix: 39 students across 407 administered quiz items. Responses were coded as +1 for correct, -1 for incorrect, and 0 for not attempted or not shown to the student. Only +1 and -1 were treated as attempts. The matrix contains 8,209 attempted responses.
  • Question bank: 586 labelled items tagged by Component, Module, skill, and Bloom level. Of these, 407 items appeared in the response matrix for this cohort.
  • Performance report: the course’s summary outputs, including model-based ability, achievement score, attendance, skill mastery categories, and final course classification for ranked students. The report also documents that AI use during quizzes made ability a combined indicator of cognitive handling, difficulty exposure, and AI-supported problem-solving behaviour rather than a pure measure of independent memory.

Classroom-Scale Bayesian Knowledge Tracing Method

The Bayesian Knowledge Tracing (BKT) implementation did not estimate all parameters freely from the classroom data. Although data-driven estimation can reduce reliance on fixed assumptions, it requires sufficiently long response sequences to distinguish genuine learning from occasional guessing or slipping. In this study, the per-student and per-skill sequences were too short to support stable estimation, particularly for the guess and slip parameters. The model therefore used transparent, instructor-defined adjustments that could be interpreted and defended in relation to the assessment context.

Expectation-maximisation is important in BKT because the learner’s mastery state cannot be observed directly. The model sees only a sequence of correct and incorrect responses, each of which may result from mastery, guessing, or slipping. Expectation-maximisation provides a systematic way to use these observed responses to estimate the hidden mastery process and the associated parameters P(L0), P(T), P(S), and P(G) (Dempster et al., 1977). In hidden Markov models, the related Baum-Welch procedure performs this estimation by repeatedly refining the probabilities of hidden state transitions until they best explain the observed sequence (Rabiner, 1989). Its significance here is therefore methodological:

Expectation-maximisation offers a principled benchmark for determining whether BKT parameters are supported by the response data rather than imposed solely through judgement.

However, reliable estimation depends on adequate response sequences. Because the present classroom dataset contained short per-skill sequences, full expectation-maximisation estimation could produce unstable or implausible parameter values. It was therefore reserved for future robustness checks, while the main analysis used transparent classroom-scale adjustments suited to the available evidence.

Response coding and item accuracy

A correct response was coded as +1, an incorrect response as -1, and a non-attempt/not-shown response as 0. Accuracy was therefore calculated only over attempted responses. This is important because a non-attempt does not provide the same evidence as an incorrect answer. In the BKT update, non-attempts were tracked as engagement evidence but were not treated as wrong observations.

Raw item accuracy was smoothed using a pseudo-count correction so that items with perfect or zero observed accuracy did not produce extreme difficulty values. The implemented formula was:

c + Correct Responses(2 ×c) + Attempts\frac{c\ +\ Correct\ Responses}{(2\ \times c)\ +\ Attempts}

Where c = 0.4

This formula adds a small stabilising correction to both the correct and incorrect sides of the item response count. It is useful in a classroom dataset because an item answered correctly by all attempting students should not automatically be treated as having zero difficulty, particularly when attempt counts vary across questions.

Adjusted Guess Probability

In standard BKT, P(G) is described as the probability that an unmastered learner guesses correctly. In the present setting, this interpretation is too narrow. Under AI-assisted or open-book conditions, P(G) is better interpreted as the probability of a correct response without independent mastery. This includes random guessing, recognition, search, AI-supported answer generation, or surface-level verification.

P(Guess) Adjusted=0.3×ItemAccuracyP(Guess)\ Adjusted = 0.3\times Item Accuracy

This means items that are easier to answer are assigned a higher probability of being answered correctly without mastery. Therefore, a correct response on an easy item provides weaker evidence of independent mastery. In contrast, a correct response on a difficult item provides stronger evidence because fewer students answered it correctly.

Item difficulty

Item difficulty was not treated as a single raw statistic because observed accuracy in this study reflected performance under an AI-permitted, open-book quiz condition rather than pure independent mastery. In such a setting, a high correct rate may indicate that an item was genuinely easy. It may also reflect answer retrieval, AI-supported reasoning, peer assistance, or limited diagnostic separation between stronger and weaker students. Therefore, raw difficulty was treated only as an initial behavioural signal and was smoothed and calibrated before being used in the Bayesian Knowledge Tracing (BKT) model. This approach aligns with BKT extensions that use contextual item information to adjust slip and guess probabilities (de Baker et al., 2008). It also aligns with KT-IDEM, which explicitly incorporates item difficulty into knowledge tracing (Pardos & Heffernan, 2011). This calibration was necessary because it was inconsistent with Bloom’s hierarchy as students performed evenly across levels. Therefore, higher-order items could not be assumed to be harder in practice.

Thus, the implemented sequence began with pseudo-count smoothed accuracy:

Accuracy=c+Correct Responses(2×c)+AttemptsAccuracy = \frac {c + Correct\ Responses} {(2 \times c) + Attempts}

Here, Correct Responses refers to responses coded as correct (1), and Attempts includes correct (1) and incorrect (-1) responses. Non-attempted (0) or not-shown items were excluded from the denominator. Difficulty was then defined as the inverse of this smoothed accuracy:

Difficulty=1AccuracyDifficulty = 1 – Accuracy

This difficulty value was not used directly. Because item exposure and attempt counts varied, the model next applied an attempt-confidence adjustment. Observed item difficulty was trusted more when there was stronger attempt evidence and pulled toward the overall item-bank difficulty when the attempt evidence was weaker:

SmoothenedDifficulty=AttemptConfidence×Difficulty+(1AttemptConfidence)×OverallDifficultySmoothened Difficulty = Attempt Confidence × Difficulty + (1 – Attempt Confidence) × Overall Difficulty
Where; AttemptConfidence=AttemptsAttempts+(5×c×TotalAttemptRate)Where;\ Attempt Confidence = \frac {Attempts} {Attempts + (5 \times c \times Total Attempt Rate)}

And c = 0.4

This step is important because an item should not be treated as genuinely difficult merely because a small number of students attempted it and performed poorly. Conversely, when many students attempted an item, the observed difficulty was treated as more reliable. The smoothened difficulty therefore represents a confidence-weighted behavioural difficulty estimate rather than a simple inverse of pass rate.

The smoothened difficulty was then translated into a difficulty band. Lower values of smoothened difficulty received lower band values, while progressively more difficult items received higher band values. This banded form made the difficulty estimate more stable and interpretable for classroom use. The final adjusted difficulty combined this banded value with a small percentile-based correction:

AdjustedDifficulty=0.90×ItemDifficultyBand+0.10×ItemDifficultyPercentileAdjusted Difficulty = 0. 90 \times Item Difficulty Band + 0.1 0 \times Item Difficulty Percentile

The 90:10 weighting made absolute difficulty bands the dominant signal while allowing a small contribution from the item’s relative position within the quiz bank. This design was chosen because only percentile-based difficulty can be misleading in AI-assisted quizzes. An item may be relatively difficult within the quiz bank but still easy in absolute terms, or it may appear easy because students can use AI, search, peer support, or procedural templates.

Adjusted slip and learning probabilities

The adjusted difficulty value was then used to scale the slip and learning probabilities. This was necessary because quiz responses were not generated under controlled closed-book conditions, and a uniform error assumption would be too restrictive. In standard BKT, the slip parameter represents the probability that a learner has mastered a skill but still answers incorrectly. The learning parameter represents a different probability. It describes the transition from an unmastered state to a mastered state after a learning opportunity (Corbett & Anderson, 1994). In this study, both parameters were adjusted using calibrated item difficulty. More difficult items are more likely to produce errors, even among students with partial or substantial mastery. This is especially true when a task requires interpretation, multi-step reasoning, or tool-mediated execution. Difficult items also provide stronger opportunities for practice, feedback, and conceptual adjustment than very easy items. This treatment is consistent with contextual extensions of BKT. These extensions show that item or task conditions can improve estimates of slip and guess probability. They avoid assuming fixed behaviour across all observations (de Baker et al., 2008). The treatment is also consistent with KT-IDEM. This model demonstrates the importance of explicitly modelling item difficulty in knowledge tracing (Pardos & Heffernan, 2011). Therefore, the model used adjusted difficulty to scale both slip and learning. Higher calibrated difficulty increased the allowance for slip. It also increased the estimated learning opportunity after an attempt.

Slip was adjusted as follows:

P(Slip) Adjusted=0.2×AdjustedDifficultyP(Slip)\ Adjusted = 0.2 \times AdjustedDifficulty

Learning was adjusted as follows:

P(Learning) Adjusted=0.4×AdjustedDifficultyP(Learning)\ Adjusted = 0.4 \times AdjustedDifficulty

The interpretation is straightforward. Items with higher calibrated difficulty carry higher slip risk because students who broadly understand the skill may still answer demanding items incorrectly. The same items also carry higher learning opportunity because they are more likely to require meaningful engagement and correction. The learning multiplier was set higher than the slip multiplier to reflect the pedagogical assumption that difficulty should contribute more to learning progression than to penalty alone. Bloom labels were retained for design and validation, but the final slip and learning adjustments were driven by calibrated item difficulty.

Mastery update and pedagogical classification

After each attempted response, the Bayesian Knowledge Tracing (BKT) update used the adjusted guess, slip, and learning probabilities to update the learner’s skill-level mastery probability. Final mastery was interpreted as the latest available mastery estimate for a student-skill combination after the relevant sequence of attempted items. These probabilities were then translated into pedagogical categories.

Final mastery rangeStatusPedagogical interpretation
< 0.30 Relearn Weak evidence of mastery. The concept should be retaught from the foundation.
0.30 to < 0.60 Practice Partial but unstable mastery. The student needs guided practice and feedback.
0.60 to < 0.80 Consolidate Near mastery. The student needs mixed practice, explanation, and reinforcement before advancement.
>= 0.80 Advance Strong mastery evidence. The student is ready for application, integration, or extension tasks.

The mastery-status classification was not used only as a descriptive label. It became a practical filtering mechanism for subsequent assessment and placement interpretation. After the Bayesian Knowledge Tracing (BKT) update, student-skill combinations were grouped into Relearn, Practice, Consolidate, and Advance. The Consolidate and Advance categories were treated as evidence of usable mastery, while Relearn and Practice indicated areas requiring further teaching support. This helped reduce the VIVA focus from the entire quiz bank to a smaller set of skills where the student had already shown sufficient learning evidence. In the VIVA, these skills could then be tested more deeply through explanation, justification, and applied questioning.

Did the Bayesian Knowledge Tracing (BKT) Assumptions Behave Pedagogically?

The validation strategy was not intended to establish the adjusted parameters as definitive psychological constructs. Rather, it assessed whether the parameter assumptions produced patterns consistent with pedagogical expectations and whether quiz correctness provided sufficiently diagnostic evidence under AI-assisted assessment conditions. Three complementary diagnostic procedures were employed.

DiagnosticWhy it was usedWhat it helps infer
Discrimination index To test whether items separate stronger from weaker students. Whether high item accuracy is diagnostically meaningful or merely high answerability.
Bloom and difficulty calibration To test whether intended cognitive demand matches observed item behaviour. Whether Bloom labels can be treated as reliable difficulty proxies under AI-assisted quizzes.
Correlation with ability and achievement To compare quiz accuracy with engagement, model-based ability, and broader assessment evidence. Whether quiz correctness alone aligns with demonstrated competency across reports, VIVA, and capstone work.

Discrimination index and warning pattern

The warning-pattern item was defined as an item with a high pass rate but poor discrimination. Such an item appears easy or successful at the surface level because many students answer it correctly, but it does not meaningfully separate stronger students from weaker students. In an AI-assisted assessment environment, this pattern is important because high correctness may reflect answerability, resource use, peer support, or weak item design rather than independent mastery.

For each item with adequate response data, a classical discrimination index D was computed. A rest-score was first calculated for each student by summing correct responses across all other items, excluding the item being diagnosed. Students were then ranked by this rest-score, and the top and bottom 27% groups were compared. With 39 students, this yielded groups of 11 students each. The discrimination index was calculated as:

D=Proportion correct in upper group  Proportion correct in lower groupD = Proportion\ correct\ in\ upper\ group\ -\ Proportion\ correct\ in\ lower\ group

Items were classified as good discrimination when D >= 0.30, marginal discrimination when 0.10 <= D < 0.30, and poor discrimination when D < 0.10. The warning pattern was defined as pass rate >= 0.85 combined with D < 0.10. This pattern does not prove AI use. Its meaning is narrower and more defensible:

Many students answered the item correctly, but stronger students did not perform meaningfully better than weaker students. Such an item has low diagnostic value under the observed assessment condition.

Possible explanations include AI assistance, peer help, over-teaching, weak item design, excessive easiness, or misalignment between the item and its tagged skill.

Bloom-level calibration

Bloom’s taxonomy was included because the question bank labelled items by intended cognitive demand. In theory, higher Bloom levels should place greater cognitive demands on students. In practice, especially under AI-assisted assessment, intended cognitive demand may not translate into observed difficulty. An Evaluate or Create item may become easy if students can obtain structured support, while a lower-level procedural item may remain difficult if it requires precise execution. The Bloom analysis therefore tested whether designed cognitive demand behaved like actual item difficulty in this cohort.

Correlation with course outcomes

The course did not rely only on quiz correctness. It combined quiz evidence with attendance, reporting, feasibility analysis, capstone work, and VIVA. This allowed quiz accuracy to be compared with broader indicators of demonstrated competency. If quiz accuracy were a strong proxy for mastery, it should align clearly with the achievement composite. If correctness was contaminated or ambiguous, quiz accuracy should be weaker than engagement volume or broader applied evidence.

What the Adjusted Bayesian Knowledge Tracing Model Revealed?

Of the 407 administered items, 366 had computable discrimination values under the study criteria. Among these items, 169 (46.2%) showed good discrimination, 86 (23.5%) showed marginal discrimination, and 111 (30.3%) showed poor discrimination. The correlation between item pass rate and discrimination was weak and only modestly positive (Pearson r = 0.18, p < 0.001). This indicates that item difficulty and item discrimination behaved largely as separate dimensions in this quiz bank.

Figure 1: Discrimination profile of the item bank

Where Good: D >= 0.30; Marginal: 0.10 <= D < 0.30; Poor: D < 0.10;

The warning pattern – high pass rate combined with poor discrimination – appeared in 56 of the 366 items with computable discrimination, or 15.3%. These items are important for instructional review because they look successful on raw pass rate but weak as mastery diagnostics. In an AI-assisted classroom, such items should not be removed automatically; they should be reviewed for wording, skill alignment, AI-solvability, and whether they should carry a higher guess adjustment.

Share of items in each module
Warning pattern mean pass rate & discrimination

It was found that warning-pattern items were not distributed evenly. Excel-module items, which were largely mechanical and procedural, showed roughly 2.4 times the warning share of Business Requirement items. The warning pattern was concentrated in skills such as Pivot/Unpivot, Table governance, Table design, Data aggregation, and Lookup functions. By contrast, Requirement Engineering and Decision System items had lower warning shares and higher mean discrimination. This supports a differentiated parameter interpretation:

Procedural items that are easily answered through formulas, templates, AI prompts, or shared examples may require a higher correct-without-mastery assumption than judgement-oriented items requiring interpretation of a business scenario.

Share of items in each Bloom level
Bloom-level difficulty calibration

Neither pass rate nor discrimination showed a significant monotonic relationship with Bloom level. Spearman correlation was -0.04 (p = 0.48) for pass rate and 0.06 (p = 0.24) for discrimination. Create-level items were, if anything, slightly easier on average than Analyse-level items. This result is important because it shows that Bloom labels should not be treated as direct difficulty estimates under AI-assisted assessment. Bloom level remains useful as a marker of intended cognitive demand, but observed difficulty must be calibrated empirically.

This result is also consistent with the course report, which noted that students performed evenly across levels and that the difficulty calibration of the item bank was inconsistent with Bloom’s hierarchy. The finding therefore strengthens the choice to calibrate observed item accuracy through smoothened and adjusted difficulty rather than using Bloom level alone.

Quiz accuracy, engagement, ability, and achievement

The course’s model-based ability estimate correlated more strongly with quiz engagement volume than with quiz accuracy. This indicates that the amount of response evidence generated by a student carried substantial weight in the ability estimate. Meanwhile, quiz accuracy had only a weak and non-significant relationship with the 40-point achievement composite, which included attendance, reporting, feasibility, capstone, and VIVA evidence. In practical terms, students who performed well on quizzes did not necessarily demonstrate proportionately stronger applied achievement once harder-to-assist outputs were considered.

Thus, in the context of AI-assisted assessment, quiz correctness is useful but cannot independently represent mastery. Teachers should therefore interpret accuracy alongside engagement volume, item difficulty and discrimination, and applied evidence from VIVA, reporting, and capstone performance.

Interpreting Bayesian Knowledge Tracing as a Pedagogical Diagnostic

The findings support using BKT as a pedagogical diagnostic, but only after its parameters are reinterpreted for the classroom context. A fixed, small guess probability is difficult to defend when students have access to AI or open resources. In this study, P(G) was therefore operationalised as correct response without independent mastery and adjusted using smoothed item accuracy. This made correct responses on highly answerable items less powerful as mastery evidence.

Similarly, slip and learning were adjusted using calibrated item difficulty. This gave the model a more educationally plausible structure:

Difficult items allow knowledgeable students to slip, but they also create richer learning opportunities.

The resulting mastery categories then converted probability estimates into instructional actions. A Relearn status means the skill should be retaught; Practice means guided exercises are needed; Consolidate means the student is close but needs reinforcement; Advance means the student can move toward applied or integrative work.

The discrimination analysis adds a second layer of defensibility. It shows that raw pass rate and item discrimination are not the same thing. Some items may be easy and still discriminating; others may be easy but non-diagnostic. The latter group is especially important in AI-assisted quizzes because a high pass rate can be produced by genuine mastery, resource use, shared answers, poor item design, or over-scaffolding. The warning-pattern diagnostic gives the instructor a practical way to identify which items require review before they are used to infer mastery.

The Bloom analysis shows that intended cognitive demand did not map neatly onto observed difficulty. This does not invalidate Bloom tagging; rather, it shows that Bloom should be used as one design signal, not as a substitute for empirical calibration. In AI-assisted learning environments, the actual difficulty of an item depends not only on the cognitive process named by the instructor but also on how easily the response can be generated, verified, or copied using external support.

Finally, the weak relationship between quiz accuracy and broader achievement reinforces the course design. The report, VIVA, feasibility work, and capstone outputs supplied evidence that was less reducible to item-level correctness. For Business Analytics training, this matters because the desired outcome is not only answering quiz items but also framing problems, designing models, interpreting evidence, and defending recommendations.

What the Model Could and Could Not Establish?

  • No direct AI-use disclosure was available. The response matrix records correctness, not whether a response came from independent reasoning, notes, AI assistance, or peer collaboration. AI-contamination inferences are therefore indirect.
  • The warning pattern is diagnostic, not causal. High pass rate with low discrimination may reflect AI assistance, but it may also reflect a poorly worded item, a genuinely easy item, ceiling effects, weak skill tagging, or over-teaching.
  • The ranked outcome analysis used available profile data for a subset of students. The excluded students had lower engagement, so relationships between quiz accuracy and final outcomes should not be generalized to the entire enrolled cohort without accounting for selection.
  • The study did not perform full expectation-maximisation-fitted Bayesian Knowledge Tracing (BKT). This was intentional because per-skill classroom sequences were short, but future work should compare the adjusted classroom model with constrained per-skill expectation-maximisation fits for high-volume skills.
  • The question count differs between the report dashboard and the response matrix. The empirical analysis uses the response matrix, but future reporting should reconcile dashboard filters and item administration counts.
  • Difficulty-band thresholds and multiplier values were instructor-defined. Although pedagogically transparent, they remain modelling assumptions and should be tested through sensitivity analysis.

Using BKT to Decide Who Should Relearn, Practice, Consolidate, or Advance?

This article examined an adjusted classroom-scale Bayesian Knowledge Tracing (BKT) implementation in an AI-assisted Business Analytics course. The model treated quiz responses as useful but imperfect evidence of mastery. It adjusted guess using smoothed item answerability, adjusted slip and learning using calibrated item difficulty, and converted final mastery probabilities into pedagogical categories.

The empirical findings support this approach. A meaningful minority of items showed the high-pass/low-discrimination warning pattern, especially in procedural skills more vulnerable to resource-assisted correctness. Bloom labels did not reliably predict observed difficulty or discrimination, confirming the need for empirical calibration. Ability estimates aligned more strongly with engagement volume than with accuracy, and quiz accuracy alone was a weak predictor of broader course achievement.

BKT can be valuable at classroom scale, but only if its parameters are interpreted as pedagogical beliefs rather than as automatically fitted truths. In AI-assisted assessment, correctness must be weighted by answerability, difficulty, cognitive demand, discrimination, and external evidence of application. Used in this way, BKT becomes less a scoring mechanism and more a structured tool for deciding who should relearn, practice, consolidate, or advance.

References

  • Abdelrahman, G., Wang, Q., & Nunes, B. (2023). Knowledge tracing: A survey. ACM Computing Surveys, 55 (11), Article 224. https://doi.org/10.1145/3569576
  • Chan, C. K. Y. (2023). A comprehensive AI policy education framework for university teaching and learning. International Journal of Educational Technology in Higher Education, 20 (1), Article 38. https://doi.org/10.1186/s41239-023-00408-3
  • Corbett, A. T., & Anderson, J. R. (1994). Knowledge tracing: Modeling the acquisition of procedural knowledge. User Modeling and User-Adapted Interaction, 4 (4), 253-278. https://doi.org/10.1007/BF01099821
  • de Baker, R. S. J., Corbett, A. T., & Aleven, V. (2008). More accurate student modeling through contextual estimation of slip and guess probabilities in Bayesian knowledge tracing. In B. P. Woolf, E. Aimeur, R. Nkambou, & S. Lajoie (Eds.), Intelligent tutoring systems: 9th International Conference, ITS 2008 (pp. 406-415). Springer. https://doi.org/10.1007/978-3-540-69132-7_44
  • Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society: Series B (Methodological), 39 (1), 1-38. https://doi.org/10.1111/j.2517-6161.1977.tb01600.x
  • Käser, T., Klingler, S., Schwing, A. G., & Gross, M. H. (2017). Dynamic Bayesian networks for student modeling. IEEE Transactions on Learning Technologies, 10 (4), 450-462. https://doi.org/10.1109/TLT.2017.2689017
  • Kelley, T. L. (1939). The selection of upper and lower groups for the validation of test items. Journal of Educational Psychology, 30 (1), 17-24. https://doi.org/10.1037/h0057123
  • Khajah, M., Lindsey, R. V., & Mozer, M. C. (2016). How deep is knowledge tracing? In T. Barnes, M. Chi, & M. Feng (Eds.), Proceedings of the 9th International Conference on Educational Data Mining (pp. 94-102). International Educational Data Mining Society.
  • Lee, J. I., & Brunskill, E. (2012). The impact of individualizing student models on necessary practice opportunities. In K. Yacef, O. Zaiane, A. Hershkovitz, M. Yudelson, & J. Stamper (Eds.), Proceedings of the 5th International Conference on Educational Data Mining (pp. 118-125). International Educational Data Mining Society.
  • Pardos, Z. A., & Heffernan, N. T. (2010). Modeling individualization in a Bayesian networks implementation of knowledge tracing. In P. De Bra, A. Kobsa, & D. Chin (Eds.), User modeling, adaptation, and personalization (pp. 255-266). Springer. https://doi.org/10.1007/978-3-642-13470-8_24
  • Pardos, Z. A., & Heffernan, N. T. (2011). KT-IDEM: Introducing item difficulty to the knowledge tracing model. In J. A. Konstan, R. Conejo, J. L. Marzo, & N. Oliver (Eds.), User modeling, adaptation, and personalization (pp. 243-254). Springer. https://doi.org/10.1007/978-3-642-22362-4_21
  • Perkins, M., Furze, L., Roe, J., & MacVaugh, J. (2024). The Artificial Intelligence Assessment Scale (AIAS): A framework for ethical integration of generative AI in educational assessment. Journal of University Teaching and Learning Practice, 21 (6). https://doi.org/10.53761/q3azde36
  • Project Guru. (2026). Business Analytics Training performance report: GIET, Sonipat, July 2026 .
  • Rabiner, L. R. (1989). A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE, 77 (2), 257-286. https://doi.org/10.1109/5.18626
  • Wang, W., Ma, H., Zhao, Y., Li, Z., & He, X. (2022). Tracking knowledge proficiency of students with calibrated Q-matrix. Expert Systems with Applications, 192 , Article 116454. https://doi.org/10.1016/j.eswa.2021.116454
  • Yudelson, M. V., Koedinger, K. R., & Gordon, G. J. (2013). Individualized Bayesian knowledge tracing models. In H. C. Lane, K. Yacef, J. Mostow, & P. Pavlik (Eds.), Artificial intelligence in education (pp. 171-180). Springer. https://doi.org/10.1007/978-3-642-39112-5_18
NOTES

I am an interdisciplinary educator, researcher, and technologist with over a decade of experience in applied coding, educational design, and research mentorship in fields spanning management, marketing, behavioral science, machine learning, and natural language processing. I specialize in simplifying complex topics such as sentiment analysis, adaptive assessments and data visualizatiion. My training approach emphasizes real-world application, clear interpretation of results and the integration of data mining, processing, and modeling techniques to drive informed strategies across academic and industry domains.

Discuss